I have a dataset consisting of a two-dimensional array (flexData[5][2]). I have this defined in my Powershell script as follows:
class flexData {
[DateTime]$dateTime
[string]$firmwareVersion
[string[][]]$flexData
}
$flexObj = [flexData]#{dateTime = $(Get-Date); firmwareVersion = 'u031C'; flexData = #(#(0, 1), #(320, 17), #(45, 36), #(0, 0))}
The problem with this is that the output object that ConvertTo-Json spits out is hard to read:
{
"dateTime": "2021-10-11T13:58:25.0937842+02:00",
"firmwareVersion": "u031C",
"flexData": [
[
"0",
"1"
],
[
"320",
"17"
],
[
"45",
"36"
],
[
"0",
"0"
]
]
}
Is there a way to instead of using a single key name and two-dimensional arrays, to instead convert this to flexData0, flexData1 ... flexData4 and keep my actual data as single-dimensional arrays? I could obviously do this by manually defining my class as:
class flexData {
[DateTime]$dateTime
[string]$firmwareVersion
[string[]]$flexData0
[string[]]$flexData1
[string[]]$flexData2
[string[]]$flexData3
[string[]]$flexData4
}
But is there a smarter way of doing this? Especially since I would also like to make a third-dimension of my array to store multiple iterations of flexData?
You could add a constructor to your flexData class that creates an object from the top-level array instead:
class flexData {
[DateTime]$dateTime
[string]$firmwareVersion
[psobject]$flexData
flexData([DateTime]$dateTime, [string]$firmwareVersion, [string[][]]$flexData){
$this.dateTime = $dateTime
$this.firmwareVersion = $firmwareVersion
# Create object from nested array
$dataProperties = [ordered]#{}
for($i = 0; $i -lt $flexData.Length; $i++){
$dataProperties["$i"] = $flexData[$i]
}
$this.flexData = [pscustomobject]$dataProperties
}
}
Now, the individual outer array items will be listed as properties named 0 through (N-1):
PS ~> $data = [flexData]::new($(Get-Date), 'u031C', #(#(0, 1), #(320, 17), #(45, 36), #(0, 0)))
PS ~> $data |ConvertTo-Json
{
"dateTime": "2021-10-11T14:21:48.4026882+02:00",
"firmwareVersion": "u031C",
"flexData": {
"0": [
"0",
"1"
],
"1": [
"320",
"17"
],
"2": [
"45",
"36"
],
"3": [
"0",
"0"
]
}
}
Related
I've used this question to get me started. However, I have the following JSON data that has multiple column headers and rows that contain multiple sets of data.
{
"columnHeaders": {
"dimensions": [
"year_month",
"tag1",
"computer_name",
"account_name",
"resource_name",
"category3",
"tag8",
"tag11",
"db_engine"
],
"metrics": [
{
"name": "usage_amount",
"dataType": "float"
},
{
"name": "pretax_cost",
"dataType": "currency"
},
{
"name": "actial_cost",
"dataType": "currency"
}
]
},
"rows": [
{
"dimensions": [
"2022-04",
"(not set)",
"server1",
"account1",
"server1_bios_name",
"undefined",
"(not set)",
"(not set)",
"SQLServer"
],
"metrics": [
{
"count": "1",
"max": "0.52",
"min": "0.10",
"sum": "0.52"
},
{
"count": "1",
"max": "-22.24",
"min": "-22.24",
"sum": "-22.24"
},
{
"count": "1",
"max": "1.26",
"min": "0",
"sum": "1.52"
}
]
}
]
}
If I use the last suggestion, I can get the dimensions fine, but that I need to the "sum" value for each metric.
$A = Get-Content 'C:\Temp\sample.json' | ConvertFrom-Json
$Rows =
ForEach($Row in $A.rows )
{
$TmpHashCol = [Ordered]#{}
$TmpHashMet = [Ordered]#{}
For($i = 0; $i -lt $Row.dimensions.Length; ++$i )
{
$TmpHashCol.Add($A.columnHeaders.dimensions[$i],$Row.dimensions[$i])
#For($s = 0; $s -lt $Row.metrics.length; ++$s )
# {
$TmpHashMet.Add($A.columnHeaders.metrics.name[$i],$Row.metrics.sum[$i])
#$TmpHashMet.Add($A.columnHeaders.metrics[$i],$Row.metrics.sum[$i])
# }
}
For the '$TmpHashMet' I get the error: Exception calling "Add" with "2" argument(s): "Key cannot be null. Yet, when I try to specify the name it doesn't like that either.
This is what I would like for the data to look like when complete:
year_month : 2022-04
tag1 : (not set)
computer_name : server1
account_name : account1
resource_name : server1_bios_name
category3 : undefined
tag8 : (not set)
tag11 : (not set)
db_engine : SQLServer
usage_amount : 0.52
pretax_cost : -22.24
actial_cost : 1.52
Many thanks in advance!!!
This is one way you could do it, you need a new inner loop for the Values on .metrics.sum, I have also modified your code a bit so it works in case the Json comes with more than 1 row.
$headers = $json.columnHeaders.dimensions
$metrics = $json.columnHeaders.metrics.name
foreach($row in $json.rows) {
$out = [ordered]#{}
for($i = 0; $i -lt $row.dimensions.Count; $i++) {
$out[$headers[$i]] = $row.dimensions[$i]
}
for($i = 0; $i -lt $metrics.Count; $i++) {
$out[$metrics[$i]] = $row.metrics.sum[$i]
}
[pscustomobject] $out
}
The output should look like your desired one:
year_month : 2022-04
tag1 : (not set)
computer_name : server1
account_name : account1
resource_name : server1_bios_name
category3 : undefined
tag8 : (not set)
tag11 : (not set)
db_engine : SQLServer
usage_amount : 0.52
pretax_cost : -22.24
actial_cost : 1.52
I have a Perl file code state.pl where I am trying to retrieve full name of State based on State code from Hash of Arrays. Following is the code:
my $all_state_details = {
IN => [
[
"AP",
"Andhra Pradesh"
],
[
"AN",
"Andaman and Nicobar Islands"
],
[
"AR",
"Arunachal Pradesh"
],
],
US => [
[
"AL",
"Alabama"
],
[
"AK",
"Alaska"
],
[
"AS",
"American Samoa"
],
],
};
my $state = 'AN';
my $country = 'IN';
my #states = $all_state_details->{$country};
my #state_name = grep { $_->[0] eq $state } #states;
print #state_name;
When I run the script, I get the blank output
I want the output as just:
Andaman and Nicobar Islands
The #{ ... } dereference operation is necessary to convert the array reference in $all_state_details->{$country} into an array suitable for use with grep.
print map { $_->[1] }
grep { $_->[0] eq $state } #{$all_state_details->{$country}};
The right way to do this kind of lookup is with a hash of hashes.
e.g.
%all_state_details = (
IN => {
"AP" => "Andhra Pradesh",
"AN" => "Andaman and Nicobar Islands",
},
);
Then you just do a direct lookup.
print $all_state_details{IN}{AN};
Read https://perldoc.perl.org/perldsc.html#HASHES-OF-HASHES
HTH
I want to use jq to remove all objects with a given name from all arrays in the input data. For example deleting "Name1" from this:
{
"Category1": [
{
"name": "Name1",
"desc": "Desc1"
},
{
"name": "Name2",
"desc": "Desc2"
}
],
"Category2": [
{
"name": "Name1",
"desc": "Desc1"
},
{
"name": "Name3",
"desc": "Desc3"
}
],
"Category3": [
{
"name": "Name4",
"desc": "Desc4"
}
]
}
Should yield this:
{
"Category1": [
{
"name": "Name2",
"desc": "Desc2"
}
],
"Category2": [
{
"name": "Name3",
"desc": "Desc3"
}
],
"Category3": [
{
"name": "Name4",
"desc": "Desc4"
}
]
}
I haven't worked with jq, or indeed JSON, much and after several hours of googling and experimenting I haven't been able to figure it out. How would I do this?
The closest I managed was this:
cat input | jq 'keys[] as $k | .[$k] |= map( select( .name != "Name1"))'
This does filter each of the arrays but returns the result as three separate objects and this is not what I want.
If the structure of your input JSON is always as seen on your example, try this:
map_values(map(select(.name != "Name1")))
Here is a solution that will remove all objects with the specified name, wherever they occur. It uses the generic function walk/1,
which is a built-in in versions of jq > 1.5, and can therefore be omitted if your jq includes it, but there is no harm in including it redundantly, e.g. in a jq script.
# Apply f to composite entities recursively, and to atoms
def walk(f):
. as $in
| if type == "object" then
reduce keys[] as $key
( {}; . + { ($key): ($in[$key] | walk(f)) } ) | f
elif type == "array" then map( walk(f) ) | f
else f
end;
walk(if type == "object" and .name == "Name1" then empty else . end)
If you really only want to remove objects from arrays, then you could use:
walk(if type == "array" then map(select( type != "object" or .name != "Name1")) else . end)
Here is a solution which uses reduce and del
reduce keys[] as $k (
.
; del(.[$k][] | select(.name == "Name1"))
)
I have following JSON structure:
{
"name": "MessageEnvelope",
"type": "record",
"fields": [
{
"name": "message",
"type":
{
"name": "Message",
"type": "record",
"fields": [
....
]
}
},
{
"name": "pipeline_system",
"type": {
"type": "enum",
"name": "PipelineSystem",
"symbols": [ "enterprise", "backscoring", "compliance" ]
}
},
{
"name": "batch_id",
"type": [ "null", "string" ]
}
]
}
I am looking to sort the above JSON file as best as it could be. For example:
fields : [
{
"name": "batch_id",
"type": [ "null", "string" ]
},
...
...
{
"name": "pipeline_system",
"type": {
"type": "enum",
"name": "PipelineSystem",
"symbols": [ "backscoring", "compliance", "enterprise" ]
}
}
Like it sorts the internal arrays as well as hashes. I am trying to write following:
def sort(collection)
if collection.is_a?(Hash)
puts "Hash Object...."
if(collection["type"]=="record")
puts "record found... Type = #{collection["fields"].class}";
if collection["fields"].is_a?(Array)
puts "fields type is array...." #we can sort fields arrays on the basis of name
collection["fields"].sort_by{|arrayCollectionElement| arrayCollectionElement["name"] }
arrayCollection = collection["fields"] #this is array of hash...we can sort them on the basis of name..done above in sort by...
puts "class = #{arrayCollection.class}"
puts "sorted fields: #{arrayCollection}"
end
end #else it is again a hash
end
collection
end
but it is not sorting the fields array on the basis of names.
Appreciate any possible help!
If I properly understood the requirements:
json = '...'
require 'json'
hash = JSON.parse json
# ⇓ let’s sort the array of fields inplace
hash['fields'].sort_by! { |o| o['name'] }
hash
#⇒ {
# "fields" => [
# [0] {
# "name" => "batch_id",
# "type" => [ "null", "string" ]
# },
# [1] {
# "name" => "message",
# "type" => {
# "fields" => [],
# "name" => "Message",
# "type" => "record"
# }
# },
# [2] {
# "name" => "pipeline_system",
# "type" => {
# "name" => "PipelineSystem",
# "symbols" => [ "enterprise", "backscoring", "compliance" ],
# "type" => "enum"
# }
# }
# ],
# "name" => "MessageEnvelope",
# "type" => "record"
# }
To sort all arrays inside, one might introduce a recursive function:
def sort_arrays hash
hash.each do |_, v|
case v
when Array then v.sort!
when Hash then sort_arrays v
end
end
end
and call it on the topmost hash.
I have a file which I have read into an array, with multiple columns and I want to sort numerically by the second column. I've looked up countless similar questions and tried to directly incorporate the answers given.
here is the basic code I am using:
use strict;
use warnings;
use diagnostics;
my #arrayed = (
"\ndog", "10", "barks",
"\ncat", "20", "meows",
"\nfish", "5", "plop",
"\nant", "30", "walk",
);
print "#arrayed";
print "\n";
my #sortedarray = sort { $a->[1] <=> $b->[1] } #arrayed;
print "#sortedarray";
exit;
This gives me an error cant use string ("dog") as an array reference while strict is turned on. I tried a few other examples with other files, arrays but always get this message so I assume there must be something intrinsically wrong with my code.
could anybody more experienced shed a little light on what I'm doing wrong please, and allow me to sort by the numbered column while still maintaining the row structure.
You have a flat array, but you want an array-of-arrays:
use strict;
use warnings;
use diagnostics;
use Data::Dumper;
my #arrayed = (
["dog", "10", "barks"],
["cat", "20", "meows"],
["fish", "5", "plop"],
["ant", "30", "walk"],
);
print Dumper(\#arrayed);
my #sortedarray = sort { $a->[1] <=> $b->[1] } #arrayed;
print Dumper(\#sortedarray);
__END__
$VAR1 = [
[
'dog',
'10',
'barks'
],
[
'cat',
'20',
'meows'
],
[
'fish',
'5',
'plop'
],
[
'ant',
'30',
'walk'
]
];
$VAR1 = [
[
'fish',
5,
'plop'
],
[
'dog',
10,
'barks'
],
[
'cat',
20,
'meows'
],
[
'ant',
30,
'walk'
]
];
Your assignment does not create a multi-dimensional array:
my #arrayed = (
"\ndog", "10", "barks",
"\ncat", "20", "meows",
"\nfish", "5", "plop",
"\nant", "30", "walk",
);
You would need to use array references inside those parentheses:
my #arrayed = (
[ "\ndog", "10", "barks" ],
[ "\ncat", "20", "meows" ],
[ "\nfish", "5", "plop" ],
[ "\nant", "30", "walk" ]
);
The brackets [ ... ] create anonymous array references, which can then be stored in the array.
One of the most important things to know when debugging is what your data looks like. Doing something like what you did
print "#arrayed";
Is not very useful, since it will only show a list of the elements separated by space. Also, if you had done this with a multi-dimensional array, you would get output like this:
ARRAY(0x7fd658) ARRAY(0x7fd7f0)
Which is what array references look like when stringified. Instead, you should use the Data::Dumper module:
use Data::Dumper;
print Dumper \#arrayed;
Notice that you are printing a reference to the array. The output would be a data structure looking like what toolic has shown in his answer:
$VAR1 = [
[ ...
Note that the brackets, again, denote array references.