Sorting an array together with other array in jq - arrays

I have two arrays in a structure but between elements of arrays has logical connections. (Eg. comments[1] belongs to records[1], comments[2] belongs to records[2])
These arrays are sorted by alpabetical order of records array.
When I want to add a new element for both array then I have sort both arrays. For records array it is easy, but the elements of comments array have to move same way as each element of record.
Input:
{
"records":
[
{"content":"a"},
{"content":"z"}
],
"comments":[
{"content":"something"},
{"content":"anything"}
]
}
New elements are added to the end of arrays:
{
"records":[
{"content":"a"},
{"content":"z"},
{"content":"b"}
],
"comments":[
{"content":"something"},
{"content":"anything"},
{"content":"new element"}
]
}
Expected sorted output:
```json
{
"records":[
{"content":"a"},
{"content":"b"},
{"content":"z"}
],
"comments":[
{"content":"something"},
{"content":"new element"},
{"content":"anything"}
]
}
I tried "to_elements, map, transpose and addfunctions without any (partial) result.

Rather than appending the new elements to the arrays, you could use bsearch to find the insertion point, and then use that to perform the two insertions. Specifically, if $x is not in a sorted array, then the insertion point of $x in the array is -1 - bsearch($x).
You might find this helper function useful:
def insert($x;$i): .[:$i]+[$x]+.[$i:];
Solution
Here then is a solution for the problem at hand:
# It is assumed that (.|f) is a sorted array
def insert_into_sorted($x; f; $y; g):
def insert($x;$i): .[:$i]+[$x]+.[$i:];
(f|bsearch($x)) as $ix
| (if $ix > -1 then $ix else -1 - $ix end) as $i
| f|=insert($x; $i)
| g|=insert($y; $i) ;
insert_into_sorted( {"content": "b"}; .records;
{"content": "new element"}; .comments)

Related

Alphabetically sorting an array within an array in Ruby?

Method input:
["eat","tea","tan","ate","nat","bat"]
I have grouped each of the anagrams into their own array within an array through this method and then sorted the array by group size:
def group_anagrams(a)
a.group_by { |stringElement| stringElement.chars.sort }.values.sort_by(&:size)
end
I am struggling to figure out how to alphabetically sort the resulting arrays within the array here because as you can see nat should come before tan in the middle element of the array:
[["bat"], ["tan", "nat"], ["eat", "tea", "ate"]]
Updating with final solution:
def group_anagrams(a)
a.group_by { |stringElement| stringElement.chars.sort }.values.map(&:sort).sort_by(&:size)
end
You need to map this array and sort (map(&:sort))
def group_anagrams(ary)
ary.group_by { |s| s.chars.sort }.values.map(&:sort)
end
ary = ["eat", "tea", "tan", "ate", "nat", "bat"]
group_anagrams(ary)
# => [["ate", "eat", "tea"], ["nat", "tan"], ["bat"]]

Sort array of arrays by hash key in value and reinsert key

I'd like to reorder an array of arrays. By default, the array is sorted by the English keys:
english_sorted_terms = [
[
"A1_english", {"domains"=>[], "en"=>{"name"=>"A_english", "def"=>"A"}, "de"=>{"name"=>"Z_german", "def"=>"..." }}
],
[
"Z1_english", {"domains"=>[], "en"=>{"name"=>"Z_english", "def"=>"Z"}, "de"=>{"name"=>"A_german", "def"=>"..."}}
]
]
After sorting, 'Z1_english' should be the first element because it contains the name 'A_german'.
My other attempts do correct the 'de' sub-key but I cannot reinsert the right parent key or the key but the sorting isn't right
english_sorted_terms
.map { |term| term[1] }
.sort_by { |key| key["de"]['name'] }
english_sorted_terms
.map { |term| [term[1]].sort_by! { |key| key["de"]['name'] }.unshift term[0] }
.sort_by! { |key| key["de"]['name'] } is working but it removes the first array element.
Ruby sorting works by comparing elements and you can define the comparison criteria yourself. In this example you compare with original array item's second element's ['de']['name']. For example:
english_sorted_terms.sort { |a, b| a.second['de']['name'] <=> b.second['de']['name'] }
See https://apidock.com/ruby/Array/sort.

How to stitch together two arrays based on a common set of keys in Ruby

I have two arrays of hashes which are related by a common set of keys:
Array 1 is:
[
{0=>"pmet-add-install-module-timings.patch"},
{1=>"pmet-change-sample-data-load-order.patch"},
{2=>"pmet-configurable-recurring.patch"},
{3=>"pmet-consumers-run-staggered-by-sleep.patch"},
{4=>"pmet-dynamic-block-segment-display.patch"},
{5=>"pmet-fix-admin-label-word-breaking.patch"},
{6=>"pmet-fix-invalid-module-dependencies.patch"},
{7=>"pmet-fix-invalid-sample-data-module-dependencies.patch"},
{8=>"pmet-fix-module-loader-algorithm.patch"},
{9=>"pmet-fix-sample-data-code-generator.patch"},
{10=>"pmet-remove-id-requirement-from-layout-update-file.patch"},
{11=>"pmet-specify-store-id-for-order.patch"},
{12=>"pmet-staging-preview-js-fix.patch"},
{13=>"pmet-stop-catching-sample-data-errrors-during-install.patch"},
{14=>"pmet-visitor-segment.patch"}
]
Array 2 is:
[
{0=>"magento2-base"},
{1=>"magento/module-sample-data"},
{2=>"magento/module-configurable-sample-data"},
{3=>"magento/module-message-queue"},
{4=>"magento/module-banner"},
{5=>"magento/theme-adminhtml-backend"},
{6=>"magento/module-staging"},
{7=>"magento/module-gift-registry-sample-data"},
{8=>"magento2-base"},
{9=>"magento/module-downloadable-sample-data"},
{10=>"magento/module-catalog"},
{11=>"magento/module-sales-sample-data"},
{12=>"magento/module-staging"},
{13=>"magento2-base"},
{14=>"magento/module-customer"}
]
The hashes in these arrays have the same set of indexes, and the second array has duplicate values in keys 0, 8, and 13 as well as in 6 and 12.
My goal is to stitch the values from these two data sets together into a set of nested hashes. Wherever there is a duplicated value in Array 2, I need to collect its associated values from Array 1 and include them in a nested hash.
For example, take the magento2-base values from Array 2 and the key-associated values from Array 1. The hash structure in Ruby would look like:
hash = {
"magento2-base" => [
{0 => "m2-hotfixes/pmet-add-install-module-timings.patch"},
{8 => "m2-hotfixes/pmet-fix-module-loader-algorithm.patch"},
{13 => "m2-hotfixes/pmet-stop-catching-sample-data-errrors-during-install.patch"}
]
}
The same would hold true for any other duplicated values from Array 2, so, for example, magento/module-staging would be:
hash = {
"magento/module-staging" => [
{6 => "pmet-fix-invalid-module-dependencies.patch"},
{12 => "pmet-staging-preview-js-fix.patch"}
]
}
A larger excerpt of the resultant hash which combines these needs together would look like this:
hash = {
"magento2-base" =>
[
{0 => "m2-hotfixes/pmet-add-install-module-timings.patch"},
{8 => "m2-hotfixes/pmet-fix-module-loader-algorithm.patch"},
{13 => "m2-hotfixes/pmet-stop-catching-sample-data-errrors-during-install.patch"}
],
"magento/module-sample-data" =>
{0 => "pmet-change-sample-data-load-order.patch"},
"magento/module-configurable-sample-data" =>
{2 => "pmet-configurable-recurring.patch"},
"magento/module-message-queue" =>
{3 => "pmet-consumers-run-staggered-by-sleep.patch"}
"magento/module-staging" =>
[
{6 => "pmet-fix-invalid-module-dependencies.patch"},
{12 => "pmet-staging-preview-js-fix.patch"}
],
...
}
I used a nested loop which combines both arrays to link up the keys, and attempted to pull out the duplicates from Array 2, and was thinking I'd need to maintain both an array of the duplicate values from array 2 as well as an array of their associated values from Array 1. Then, I'd use some array merging magic to put it all back together.
Here's what I have:
found_modules_array = []
duplicate_modules_array = []
duplicate_module_hash = {}
file_collection_array = []
modules_array.each do |module_hash|
module_hash.each do |module_hash_key, module_hash_value|
files_array.each do |file_hash|
file_hash.each do |file_hash_key, file_hash_value|
if module_hash_key == file_hash_key
if found_modules_array.include?(module_hash_value)
duplicate_module_hash = {
module_hash_key => module_hash_value
}
duplicate_modules_array << duplicate_module_hash
end
found_modules_array << module_hash_value
end
end
end
end
end
In this code, files_array is Array 1 and modules_array is Array 2. found_modules_array is a bucket to hold any duplicates before pushing them into a duplicate_module_hash which would then be pushed into the duplicates_modules_array.
This solution:
Doesn't work
Doesn't take advantage of the power of Ruby
Isn't performant
EDIT
The path to the above data structure is explained in full detail in the following post: Using array values as hash keys to create nested hashes in Ruby
I'll summarize it below:
I have a directory of files. The majority of them are .patch files, although some of them are not. For each patch file, I need to scan the first line which is always a string and extract a portion of that line. With a combination of each file's name, that portion of each first line, and a unique identifier for each file, I need to create a hash which I will then convert to json and write to a file.
Here are examples:
Directory of Files:
|__ .gitkeep
|__ pmet-add-install-module-timings.patch
|__ pmet-change-sample-data-load-order.patch
First Line Examples:
File Name: `pmet-add-install-module-timings.patch`
First Line: `diff --git a/setup/src/Magento/Setup/Model/Installer.php b/setup/src/Magento/Setup/Model/Installer.php`
File Name: `pmet-change-sample-data-load-order.patch`
First Line: `diff --git a/vendor/magento/module-sample-data/etc/module.xml b/vendor/magento/module-sample-data/etc/module.xml`
File Name: `pmet-stop-catching-sample-data-errrors-during-install.patch`
First Line: `diff --git a/vendor/magento/framework/Setup/SampleData/Executor.php b/vendor/magento/framework/Setup/SampleData/Executor.php`
File Name: `pmet-fix-admin-label-word-breaking.patch`
First Line: `diff --git a/vendor/magento/theme-adminhtml-backend/web/css/styles-old.less b/vendor/magento/theme-adminhtml-backend/web/css/styles-old.less`
Example Json File:
{
"patches": {
"magento/magento2-base": {
"Patch 1": "m2-hotfixes/pmet-add-install-module-timings.patch"
},
"magento/module-sample-data": {
"Patch 2": "m2-hotfixes/pmet-change-sample-data-load-order.patch"
},
"magento/theme-adminhtml-backend": {
"Patch 3": "m2-hotfixes/pmet-fix-admin-label-word-breaking.patch"
},
"magento/framework": {
"Patch 4": "m2-hotfixes/pmet-stop-catching-sample-data-errrors-during-install.patch"
}
}
}
The problem I encountered is that while json allows for duplicate keys, ruby hashes don't, so items were removed from the json file because they were removed from the hash. To solve this, I assumed I needed to create the array structure I specified so as to keep the IDs as the consistent identifier between the files scraped and the corresponding data belonging to them so that I could put the data together in a different arrangement. Now I realize this isn't the case, so I have switched the approach to use the following:
files.each_with_index do |file, key|
value = File.open(file, &:readline).split('/')[3]
if value.match(/module-/) || value.match(/theme-/)
result = "magento/#{value}"
else
result = "magento2-base"
end
file_array << file
module_array << result
end
This yields the flat hashes that have been suggested below.
So first of all, the structure
arr1 = [
{0=>"pmet-add-install-module-timings.patch"},
{1=>"pmet-change-sample-data-load-order.patch"},
{2=>"pmet-configurable-recurring.patch"},
{3=>"pmet-consumers-run-staggered-by-sleep.patch"},
# etc
]
is a little odd. It's easier to work with as a flat hash, e.g.
h1 = {
0 => "pmet-add-install-module-timings.patch",
1 => "pmet-change-sample-data-load-order.patch",
2 => "pmet-configurable-recurring.patch",
3 =>"pmet-consumers-run-staggered-by-sleep.patch",
# etc
}
Fortunately it's quite easy to transform between the two:
h1 = arr1.reduce(&:merge)
h2 = arr2.reduce(&:merge)
From this point, Enumerable methods (in this case, the ever-useful map, group_by, and transform_values) will take you the rest of the way:
indexed_by_val = h2.
group_by { |k,v| v }.
transform_values { |vals| vals.map(&:first) }
Which gives you a map of val to indexes:
{
"magento2-base"=>[0, 8, 13],
"magento/module-sample-data"=>[1],
"magento/module-configurable-sample-data"=>[2],
# etc
}
and then we can replace those lists of indexes with the corresponding values in h1:
result = indexed_by_val.transform_values do |indexes|
indexes.map do |idx|
{ idx => h1[idx] }
end
end
which produces your desired data structure:
{
"magento2-base"=>[
{0=>"pmet-add-install-module-timings.patch"},
{8=>"pmet-fix-module-loader-algorithm.patch"},
{13=>"pmet-stop-catching-sample-data-errrors-during-install.patch"}
],
"magento/module-sample-data"=>[
{1=>"pmet-change-sample-data-load-order.patch"}
],
"magento/module-configurable-sample-data"=>[
{2=>"pmet-configurable-recurring.patch"}
],
# etc
}
I did notice that in your expected output that you specified, the values are hashes or arrays. I would recommend against this practice. It's much better to have a uniform data type for all a hash's keys and values. But, if you really did want to do this for whatever reason, it's not too difficult:
# I am not advising this approach
result2 = result.transform_values do |arr|
arr.length > 1 ? arr : arr[0]
end
By the way, I know this kind of functional programming / enumerable chaining code can be a bit hard to decipher. so I would recommend running it line-by-line for your understanding.
Assuming you're using the unified data structure I mentioned above, I would recommend calling .transform_values { |vals| vals.reduce(&:merge) } on your final result so that the values are single hashes instead of multiple hashes:
{
"magento2-base"=>{
0=>"pmet-add-install-module-timings.patch",
8=>"pmet-fix-module-loader-algorithm.patch",
13=>"pmet-stop-catching-sample-data-errrors-during-install.patch"
},
"magento/module-sample-data"=>{
1=>"pmet-change-sample-data-load-order.patch"
],
"magento/module-configurable-sample-data"=>{
2=>"pmet-configurable-recurring.patch"
},
# etc
}
Let arr1 and arr2 be your two arrays. Due to the fact that they are the same size and that for each index i, arr1[i][i] and arr2[i][i] are the values of the key i of the hashes arr1[i] and arr2[i], the desired result can be obtained quite easily:
arr2.each_with_index.with_object({}) do |(g,i),h|
(h[g[i]] ||= []) << arr1[i][i]
end
#=> {
# "magento2-base"=>[
# "pmet-add-install-module-timings.patch",
# "pmet-fix-module-loader-algorithm.patch",
# "pmet-stop-catching-sample-data-errrors-during-install.patch"
# ],
# "magento/module-sample-data"=>[
# "pmet-change-sample-data-load-order.patch"
# ],
# ...
# "magento/module-staging"=>[
# "pmet-fix-invalid-module-dependencies.patch",
# "pmet-staging-preview-js-fix.patch"
# ],
# "magento/module-customer"=>[
# "pmet-visitor-segment.patch"
# ]
# }
The fragment
h[g[i]] ||= []
is effectively expanded to
h[g[i]] = h[g[i]] || [] # *
If the hash h has no key [g[i]],
h[g[i]] #=> nil
so * becomes
h[g[i]] = nil || [] #=> []
after which
h[g[i]] << "cat"
#=> ["cat"]
(which works with "dog" as well). The above expression can instead be written:
arr2.each_with_index.with_object(Hash.new {|h,k| h[k]=[]}) do |(g,i),h|
h[g[i]] << arr1[i][i]
end
This uses the form of Hash::new that employs a block (here {|h,k| h[k]=[]}) that is called when the hash is accessed by a value that is not one of its keys.
An alternative method is:
arr2.each_with_index.with_object({}) do |(g,i),h|
h.update(g[i]=>[arr1[i][i]]) { |_,o,n| o+n }
end
This uses the form of Hash#update (aka merge!) that employs a block to determine the values of keys that are in both hashes being merged.
A third way is to use Enumerable#group_by:
arr2.each_with_index.group_by { |h,i| arr2[i][i] }.
transform_values { |a| a.map { |_,i| arr1[i][i] } }

Powershell create array of arrays

I'm trying to push data to a REST api using powershell.
http://influxdb.com/docs/v0.8/api/reading_and_writing_data.html
The server expects data like so:
[
{
"name" : "hd_used",
"columns" : ["value", "host", "mount"],
"points" : [
[23.2, "serverA", "mnt"]
]
}
]
However, I"m only able to make a json object that looks like this (notice the extra quotes):
[
{
"name" : "hd_used",
"columns" : ["value", "host", "mount"],
"points" : [
"[23.2, "serverA", "mnt"]"
]
}
]
How can I construct the data into an array of arrays without wrapping the nested array in quotes?
This works, but it isn't a nested array
$influxdata = [ordered]#{}
$influxdata.name = $hd_used
$influxdata.columns = #("value", "host", "mount")
$influxdata.points = #()
$influxdata.points += #("23.2", "serverA", "mnt")
$influxdatajson = $influxdata | ConvertTo-Json -Depth 2
This works but the inner array is actually a string.
$influxdata = [ordered]#{}
$influxdata.name = $hd_used
$influxdata.columns = #("value", "host", "mount")
$influxdata.points = #()
$influxdata.points += #('["23.2", "serverA", "mnt"]')
$influxdatajson = $influxdata | ConvertTo-Json -Depth 2
With $PSVersion.$PSVersion equal to 3.0 and your exact input I get the following when I print the $influxdatajson variable:
{
"name": "hd_used",
"columns": [
"value",
"host",
"mount"
],
"points": [
"23.2",
"serverA",
"mnt"
]
}
Which clearly isn't what you want but isn't what you said you got either.
The reason that that is the output we get is because your attempt to add the array to the existing array didn't work the way you expect because of powershell's annoying tendency to unroll arrays (I think).
If you work around that oddity by using this syntax instead:
$influxdata.points += ,#("23.2", "serverA", "mnt")
(the leading , forces an array context so that outer array gets unrolled instead of the array you are trying to add)
then I get the following output from $influxdatajson:
{
"name": "hd_used",
"columns": [
"value",
"host",
"mount"
],
"points": [
[
"23.2",
"serverA",
"mnt"
]
]
}
To complement Etan Reisner's helpful answer (whose of use unary , to create a nested array solves the problem):
PowerShell's hashtable literals are quite flexible with respect to incorporating variable references, expression, and commands, which makes for a more readable solution:
,, [ordered] #{
name = 'hd_used'
columns = 'value', 'host', 'mount'
points = , (23.2, 'serverA', 'mnt')
} | ConvertTo-Json -Depth 3
This yields:
[
{
"name": "hd_used",
"columns": [
"value",
"host",
"mount"
],
"points": [
[
23.2,
"serverA",
"mnt"
]
]
}
]
Note how the array-construction operator (,) is applied twice at the beginning of the pipeline (before [ordered]):
first to turn the ordered hashtable into a (single-item) array,
and the 2nd time to wrap that array in an outer array.
Sending the result through the pipeline makes PowerShell unwrap any collection, i.e., enumerate the collection items and send them one by one, which in this case strips away the outer array, leaving ConvertTo-Json to process the inner array, as desired.
Note that passing an array adds a level to the hierarchy, which is why the -Depth value was increased to 3 above.
Caveat: Any property whose hierarchy level is deeper than -Depth is stringified (evaluated as if placed inside "$(...)"), which in the case of an array would simply join the array elements with a space; e.g., array 23.2, 'serverA', 'mnt' would turn to a single string with literal contents 23.2 serverA mnt.
Note how the arrays above do not use syntax #(...), because it is generally not necessary to construct arrays and is actually less efficient: simply ,-enumerate the elements, and, if necessary, enclose in (...) for precedence (although #() is syntactically convenient for creating an empty array).
+ with an array as the LHS doesn't so much unwrap (unroll) its RHS, but concatenates arrays, or, to put it differently, allows you to append multiple individual items; e.g.:
$a = 1, 2
$a += 3, 4 # add elements 3 and 4 to array 1, 2 to form array 1, 2, 3, 4
Note that the use of += actually creates a new array behind the scenes, given that arrays aren't resizable.
Just to add my two pence worth. Its is worth noting that everytime you touch an array and you know is a single item array, you must use the appropriate comma syntax. This is not well highlighted in articles I found on the subject.
Take this example case I wrote for a Pester test case:
A lesson about single item arrays, every time you touch a single item array, you must use the comma op. It is not a case of set it and forget it:
Mock get-AnswerAdvancedFn -ModuleName Elizium.Loopz {
$pairs = #(, #('Author', 'Douglas Madcap Adams'));
$first = $pairs[0];
Write-Host "=== SINGLE-ITEM: pairs.count: $($pairs.Count), first.count: $($first.Count)"
([PSCustomObject]#{ Pairs = $pairs })
}
The above won't work, because of the fault assigning $pairs to Pairs even though we've used the correct syntax setting $pairs to #(, #('Author', 'Douglas Madcap Adams'))
The following fixes this issue; everytime you touch the single item array, you must use the comma syntax, otherwise you give PowerShell another chance to flatten your array:
Mock get-AnswerAdvancedFn -ModuleName Elizium.Loopz {
$pairs = #(, #('Author', 'Douglas Madcap Adams'));
$first = $pairs[0];
Write-Host "=== SINGLE-ITEM: pairs.count: $($pairs.Count), first.count: $($first.Count)"
([PSCustomObject]#{ Pairs = , $pairs })
}
My test code ended up being this:
Mock get-AnswerAdvancedFn -ModuleName Elizium.Loopz {
([PSCustomObject]#{ Pairs = , #(, #('Author', 'Douglas Madcap Adams')) })
}
Note, we had to use the comma op twice and both of those are necessary

Filling hash of multi-dimensional arrays in perl

Given three scalars, what is the perl syntax to fill a hash in which one of the scalars is the key, another determines which of two arrays is filled, and the third is appended to one of the arrays? For example:
my $weekday = "Monday";
my $kind = "Good";
my $event = "Birthday";
and given only the scalars and not their particular values, obtained inside a loop, I want a hash like:
my %Weekdays = {
'Monday' => [
["Birthday", "Holiday"], # The Good array
["Exam", "Workday"] # The Bad array
]
'Saturday' => [
["RoadTrip", "Concert", "Movie"],
["Yardwork", "VisitMIL"]
]
}
I know how to append a value to an array in a hash, such as if the key is a single array:
push( #{ $Weekdays{$weekday} }, $event);
Used in a loop, that could give me:
%Weekdays = {
'Monday' => [
'Birthday',
'Holiday',
'Exam',
'Workday'
]
}
I suppose the hash key is the particular weekday, and the value should be a two dimensional array. I don't know the perl syntax to, say, push Birthday into the hash as element [0][0] of the weekday array, and the next time through the loop, push another event in as [0][1] or [1][0]. Similarly, I don't know the syntax to access same.
Using your variables, I'd write it like this:
push #{ $Weekdays{ $weekday }[ $kind eq 'Good' ? 0 : 1 ] }, $event;
However, I'd probably just make the Good/Bad specifiers keys as well. And given my druthers:
use autobox::Core;
( $Weekdays{ $weekday }{ $kind } ||= [] )->push( $event );
Note that the way I've written it here, neither expression cares whether or not an array exists before we start.
Is there some reason that
push #{ $Weekdays{Monday}[0] }, "whatever";
isn’t working for you?

Resources