My GridDB nodes do not want to join into the same cluster - database

I have successfully started three nodes on three different Azure CentOS instances. Each node is pointing to the default notification address (239.0.0.1) and are on the same virtual network on Azure (address space 10.2.0.0/24). The nodes are all joined in on the same cluster name ("temperature" in my specific case).
Based on this, the nodes should all be in the same cluster; the problem is, when I run gs_stat, they're all clearly joined into individual clusters:
-bash-4.2$ gs_stat -u admin/password
{
"checkpoint": {
"endTime": 1542823670774,
"mode": "NORMAL_CHECKPOINT",
"normalCheckpointOperation": 1,
"pendingPartition": 0,
"requestedCheckpointOperation": 0,
"startTime": 1542823670486
},
"cluster": {
"activeCount": 1,
"clusterName": "temperature",
"clusterStatus": "MASTER",
"designatedCount": 1,
"loadBalancer": "ACTIVE",
"master": {
"address": "10.2.0.5",
"port": 10040
},
"nodeList": [
{
"address": "10.2.0.5",
"port": 10040
}
],
"nodeStatus": "ACTIVE",
"notificationMode": "MULTICAST",
"partitionStatus": "NORMAL",
"startupTime": "2018-11-21T18:06:49Z",
"syncCount": 2
},
"currentTime": "2018-11-21T18:08:33Z",
"performance": {
"backupCount": 0,
"batchFree": 0,
"checkpointFileAllocateSize": 262144,
"checkpointFileSize": 262144,
"checkpointFileUsageRate": 0,
"checkpointMemory": 0,
"checkpointMemoryLimit": 1073741824,
"checkpointWriteSize": 0,
"checkpointWriteTime": 0,
"currentCheckpointWriteBufferSize": 0,
"currentTime": 1542823713412,
"numBackground": 0,
"numConnection": 2,
"numNoExpireTxn": 0,
"numSession": 0,
"numTxn": 0,
"ownerCount": 128,
"peakProcessMemory": 72777728,
"processMemory": 72777728,
"recoveryReadSize": 262144,
"recoveryReadTime": 32,
"storeCompressionMode": "NO_BLOCK_COMPRESSION",
"storeDetail": {
"batchFreeMapData": {
"storeMemory": 0,
"storeUse": 0,
"swapRead": 0,
"swapWrite": 0
},
"batchFreeRowData": {
"storeMemory": 0,
"storeUse": 0,
"swapRead": 0,
"swapWrite": 0
},
"mapData": {
"storeMemory": 0,
"storeUse": 0,
"swapRead": 0,
"swapWrite": 0
},
"metaData": {
"storeMemory": 0,
"storeUse": 0,
"swapRead": 0,
"swapWrite": 0
},
"rowData": {
"storeMemory": 0,
"storeUse": 0,
"swapRead": 0,
"swapWrite": 0
}
},
"storeMemory": 0,
"storeMemoryLimit": 1073741824,
"storeTotalUse": 0,
"swapRead": 0,
"swapReadSize": 0,
"swapReadTime": 0,
"swapWrite": 0,
"swapWriteSize": 0,
"swapWriteTime": 0,
"syncReadSize": 0,
"syncReadTime": 0,
"totalBackupLsn": 0,
"totalLockConflictCount": 0,
"totalOtherLsn": 0,
"totalOwnerLsn": 0,
"totalReadOperation": 0,
"totalRowRead": 0,
"totalRowWrite": 0,
"totalWriteOperation": 0
},
"recovery": {
"progressRate": 1
},
"version": "4.0.0-33128 CE"
}
Is there a proper way to troubleshoot this? Is there a reason the nodes can't communicate?

It looks like you’re using GridDB with multicast. This works if you’ve got local machines but don’t seem to work on Azure (or other cloud services). The solution is to change to fixed-list mode. This will give explicit addresses for the griddb nodes to join in on as a cluster.
More info here:
https://griddb.net/en/blog/griddb-using-fixed-list-or-multicast-clustering/

Related

build grid from 2 arrays in Ruby. Javascript to Ruby

I'm trying to replicate a solution from javascript to Ruby where the idea is to build a grid from two arrays and then, running a search on both arrays, change certain elements in the grid.
I think my ruby solution is fairly similar but the grid doesn't look similar.
Any help is much appreciated
Arrays:
nums1 = [1,2,3,2,1]
nums2 = [3,2,1,4,7]
Javascript grid:
``
const dp = new Array(nums1.length + 1).fill(0).map(() => new Array(nums2.length + 1).fill(0));
console.log(dp)
output:
[
[ 0, 0, 0, 0, 0, 0 ],
[ 0, 0, 0, 0, 0, 0 ],
[ 0, 0, 0, 0, 0, 0 ],
[ 0, 0, 0, 0, 0, 0 ],
[ 0, 0, 0, 0, 0, 0 ],
[ 0, 0, 0, 0, 0, 0 ]
]
Ruby grid attempts:
1º attempt
dp = Array.new(nums1.length+1) {Array.new(nums2.length+1, 0)}
p dp
2º attempt:
dp = (0..n).map{|el| el = (0..m).to_a.map{|el| el = 0 }}
p dp
output is the same for both:
[[0, 0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 0]]
Thank you

How to sum multidimensional arrays across documents with MongoDB

Say I have a lot of the following documents:
{
_id: “abc”,
values: {
0: { 0: 999999, 1: 999999, …, 59: 1000000 },
1: { 0: 2000000, 1: 2000000, …, 59: 1000000 },
…,
58: { 0: 1600000, 1: 1200000, …, 59: 1100000 },
59: { 0: 1300000, 1: 1400000, …, 59: 1500000 }
}
}
{
_id: “def”,
values: {
0: { 0: 999999, 1: 999999, …, 59: 1000000 },
1: { 0: 2000000, 1: 2000000, …, 59: 1000000 },
…,
58: { 0: 1600000, 1: 1200000, …, 59: 1100000 },
59: { 0: 1300000, 1: 1400000, …, 59: 1500000 }
}
}
which is basically a multidimensional array of 60x60 items.
can aggregation (or any other mongodb construct) be used to easily sum the two (or more) matrixes? i.e. values[x][y] of both abc and def are summed together, and the same is done for all other elements?
Ideally the output would be a similar multidimensional array.
This answer seems to suggest this is possible with 1 dimensional array but I am not sure for multidimensional.
EDIT:
This is an example with real data in a format which is slightly different:
db.col.find({}, { _id: 0, hit: 1 })
{ "hit" : [ [ 570, 0, 630, 630, 636, 735, 672, 615, 648, 648, 618, 0 ],
[ 492, 0, 471, 471, 570, 564, 0, 590, 513, 432, 471, 477 ],
[ 387, 0, 0, 0, 0, 0, 0, 456, 0, 480, 351, 415 ],
[ 432, 528, 0, 0, 495, 509, 0, 579, 0, 552, 0, 594 ],
[ 558, 603, 594, 624, 672, 0, 0, 705, 783, 0, 756, 816 ],
[ 0, 858, 951, 1027, 0, 0, 1058, 1131, 0, 0, 1260, 1260 ],
[ 1269, 0, 1287, 0, 1326, 0, 1386, 1386, 1470, 0, 0, 0 ],
[ 1623, 0, 1695, 1764, 1671, 1671, 0, 1824, 1872, 0, 0, 0 ],
[ 1950, 1894, 2034, 2034, 0, 0, 1941, 0, 2070, 1911, 2049, 2055 ],
[ 2052, 2052, 0, 0, 0, 2085, 2007, 2073, 0, 0, 0, 1941 ],
[ 1878, 1896, 0, 1875, 0, 0, 1677, 0, 1722, 0, 1545, 0 ],
[ 0, 0, 1317, 1469, 1501, 1634, 1494, 0, 0, 1290, 0, 0 ],
[ 0, 1485, 1375, 1491, 1530, 1407, 0, 0, 0, 1611, 0, 0 ],
[ 1652, 1800, 1686, 1643, 1923, 0, 0, 0, 1737, 1604, 1797, 0 ],
[ 1842, 1806, 0, 1830, 1896, 1947, 0, 1710, 1734, 1725, 0, 0 ],
[ 0, 0, 1932, 0, 1908, 1878, 1941, 1931, 2007, 2013, 1995, 1995 ],
[ 0, 2025, 2004, 1927, 0, 0, 1939, 1835, 1962, 1863, 0, 1815 ],
[ 0, 0, 1839, 1755, 1821, 1821, 1751, 1656, 0, 0, 1467, 0 ],
[ 0, 1632, 1546, 1449, 0, 1551, 1449, 0, 0, 1554, 0, 1491 ],
[ 1463, 1411, 0, 1491, 0, 0, 1551, 1467, 0, 0, 0, 1464 ],
[ 0, 0, 1311, 0, 0, 1471, 0, 0, 1581, 0, 1368, 1368 ],
[ 1296, 0, 0, 0, 1176, 1381, 0, 1170, 1194, 1194, 1193, 1137 ],
[ 0, 1244, 1221, 1039, 0, 1041, 930, 921, 1033, 813, 0, 0 ],
[ 0, 0, 0, 1010, 0, 0, 918, 783, 0, 609, 693, 645 ] ] }
And this is the appropriate query (thanks to Veeram in the comments for fixing my code):
db.col.aggregate([
{ $project: { _id: 0, hit: 1 } },
{ $unwind: { path: "$hit", includeArrayIndex: "x" } },
{ $unwind: { path: "$hit", includeArrayIndex: "y" } },
{ $group: { _id: { x: "$x", y: "$y" }, hit: { $sum: "$hit" } } },
{ $sort: { "_id.x": 1, "_id.y": 1 } },
{ $group: { _id: "$_id.x", hit: { $push: "$hit" } } },
{ $sort: { "_id": 1 } },
{ $group: { _id: null, hit: { $push: "$hit" } } }
])
You need two operators to deal with dynamic properties: $objectToArray and $arrayToObject. To sum the values from all documents you can try to represent each x,y pair as single document (using $unwind) and then use several $group stages to get single document as a result. To get the initial order of your rows and columns you can apply $sort twice:
db.col.aggregate([
{
$project: {
values: {
$map: {
input: { $objectToArray: "$values" },
as: "obj",
in: { k: "$$obj.k", v: { $objectToArray: "$$obj.v" } }
}
}
}
},
{
$unwind: "$values"
},
{
$unwind: "$values.v"
},
{
$project: {
x: "$values.k",
y: "$values.v.k",
value: "$values.v.v"
}
},
{
$group: {
_id: { x: "$x", y: "$y" },
value: { $sum: "$value" }
}
},
{
$sort: {
"_id.y": 1
}
},
{
$group: {
_id: "$_id.x",
v: { $push: { k: "$_id.y", v: "$value" } }
}
},
{
$sort: {
"_id": 1
}
},
{
$group: {
_id: null,
values: { $push: { k: "$_id", v: "$v" } }
}
},
{
$project: {
values: {
$arrayToObject: {
$map: {
input: "$values",
as: "obj",
in: {
k: "$$obj.k",
v: { $arrayToObject: "$$obj.v" }
}
}
}
}
}
}
])
For your sample data it outputs:
{
"_id" : null,
"values" : {
"0" : {
"0" : 1999998,
"1" : 1999998,
"59" : 2000000
},
"1" : {
"0" : 4000000,
"1" : 4000000,
"59" : 2000000
},
"58" : {
"0" : 3200000,
"1" : 2400000,
"59" : 2200000
},
"59" : {
"0" : 2600000,
"1" : 2800000,
"59" : 3000000
}
}
}

Which file format is this and how to open it?

Which file format is this and how to open it?
1435708910.94,state,{"CLOSE_TIME": 1435752000, "SERVICE_STATES": {"LED1": {"USED_MINS": 96.91667685111364, "FAIL_MINS": 0, "PAYG_CREDITS_USED": 0, "LAST_USE": [1435708747.965771, 9], "LAST_AVAIL": [1435708731.860505, true], "FREE_MINS": 374.18972703218475, "CHARGE": 96.91667685111364, "LVD_MINS": 0, "NODATA_MINS": 0}, "LED2": {"USED_MINS": 96.99929953018822, "FAIL_MINS": 0, "PAYG_CREDITS_USED": 0, "LAST_USE": [1435708728.138294, 0], "LAST_AVAIL": [1435708721.04487, true], "FREE_MINS": 141.17060283819833, "CHARGE": 96.99929953018822, "LVD_MINS": 0, "NODATA_MINS": 0}, "USB1": {"USED_MINS": 0, "FAIL_MINS": 0, "PAYG_CREDITS_USED": 0, "LAST_USE": [1435708713.425554, 0], "LAST_AVAIL": [1435708702.763062, true], "FREE_MINS": 0, "CHARGE": 0, "LVD_MINS": 0, "NODATA_MINS": 0}}, "OPEN_TIME": 1435665600, "PAYG_CREDITS_USED": 0, "KERNEL_VERSION": "4.0", "LED_QUOTA_USED": 193.9159763813019, "USB_QUOTA_USED": 0, "USB_QUOTA_END": 1435665621.729028, "KERNEL_AUTHOR": "CA", "LED_QUOTA_END": 1435686998.772021}
If you remove the 1435708910.94,state, from the beginning, the rest is a JSON string. You can open the file in any text editor like Notepad, although a specialized JSON editor gives you more features like formatting it to be easier to read. You can try this online JSON Editor.
Here is the formatted version:
{
"CLOSE_TIME": 1435752000,
"SERVICE_STATES": {
"LED1": {
"USED_MINS": 96.91667685111363,
"FAIL_MINS": 0,
"PAYG_CREDITS_USED": 0,
"LAST_USE": [
1435708747.965771,
9
],
"LAST_AVAIL": [
1435708731.860505,
true
],
"FREE_MINS": 374.18972703218475,
"CHARGE": 96.91667685111363,
"LVD_MINS": 0,
"NODATA_MINS": 0
},
"LED2": {
"USED_MINS": 96.99929953018822,
"FAIL_MINS": 0,
"PAYG_CREDITS_USED": 0,
"LAST_USE": [
1435708728.138294,
0
],
"LAST_AVAIL": [
1435708721.04487,
true
],
"FREE_MINS": 141.17060283819833,
"CHARGE": 96.99929953018822,
"LVD_MINS": 0,
"NODATA_MINS": 0
},
"USB1": {
"USED_MINS": 0,
"FAIL_MINS": 0,
"PAYG_CREDITS_USED": 0,
"LAST_USE": [
1435708713.425554,
0
],
"LAST_AVAIL": [
1435708702.763062,
true
],
"FREE_MINS": 0,
"CHARGE": 0,
"LVD_MINS": 0,
"NODATA_MINS": 0
}
},
"OPEN_TIME": 1435665600,
"PAYG_CREDITS_USED": 0,
"KERNEL_VERSION": "4.0",
"LED_QUOTA_USED": 193.9159763813019,
"USB_QUOTA_USED": 0,
"USB_QUOTA_END": 1435665621.729028,
"KERNEL_AUTHOR": "CA",
"LED_QUOTA_END": 1435686998.772021
}

AngularJS: SubTotals and Totals

I have coded myself into a corner and can't figure out how to calculate the subtotals (per crop) and totals (all crops). I have the expected values hard-coded right now but need to figure out how to calculate them.
I have a plunker.
I am using budget.json to simulate the call to the database in the factory (defined in budget.js). The BudgetsController is also defined in budget.js.
The hard-coded totals begin at line 35 of budgets.js. I tried several of the LoDash methods to calculate the totals but can't seem to find the pattern that I can replicate for each crop and I know the Totals total would follow the same pattern but just using the subtotals.
Any help is appreciated!
CODE of budget.js:
(function(){
'use strict';
angular
.module('ARM')
.factory('ExpensesFactory', function ExpensesFactory(
$http, $q
) {
return {
getBudget: getBudget
};
function getBudget(id){
return $http.get('budget.json');
}
})
.controller('BudgetsController', BudgetsController);
BudgetsController.$inject = ['$scope', 'ExpensesFactory'];
function BudgetsController(
$scope, ExpensesFactory
){
ExpensesFactory.getBudget('1')
.then(function success(rsp){
var arr = rsp.data;
var flattened = _.flatten(arr);
var grped = _.groupBy(flattened, function(item) {
return item.crop;
});
$scope.uses = grped;
//TODO: Crop and Loan Budget Totals
$scope.uses.totals = [
//CORN
[
{
"arm": 178,
"dist": 197.91,
"other": 115,
"peracre": 490.91,
"calc_arm": 61837.2,
"calc_dist": 68753.934,
"calc_other": 39951,
"calc_total": 170542.134
}
],
//SOYBEANS
[
{
"arm": 145,
"dist": 69.73,
"other": 74.35,
"peracre": 289.08,
"calc_arm": 84143.5,
"calc_dist": 40464.319,
"calc_other": 43145.305,
"calc_total": 167753.124
}
],
//SORGHUM
[
{
"arm": 0,
"dist": 0,
"other": 0,
"peracre": 0,
"calc_arm": 0,
"calc_dist": 0,
"calc_other": 0,
"calc_total": 0
}
],
//WHEAT
[
{
"arm": 0,
"dist": 0,
"other": 0,
"peracre": 0,
"calc_arm": 0,
"calc_dist": 0,
"calc_other": 0,
"calc_total": 0
}
],
//COTTON
[
{
"arm": 0,
"dist": 0,
"other": 0,
"peracre": 0,
"calc_arm": 0,
"calc_dist": 0,
"calc_other": 0,
"calc_total": 0
}
],
//RICE
[
{
"arm": 0,
"dist": 0,
"other": 0,
"peracre": 0,
"calc_arm": 0,
"calc_dist": 0,
"calc_other": 0,
"calc_total": 0
}
],
//PEANUTS
[
{
"arm": 0,
"dist": 0,
"other": 0,
"peracre": 0,
"calc_arm": 0,
"calc_dist": 0,
"calc_other": 0,
"calc_total": 0
}
],
//SUGAR CANE
[
{
"arm": 0,
"dist": 0,
"other": 0,
"peracre": 0,
"calc_arm": 0,
"calc_dist": 0,
"calc_other": 0,
"calc_total": 0
}
],
//TOTALS
[
{
"arm": 0,
"dist": 0,
"other": 0,
"peracre": 0,
"calc_arm": 155999,
"calc_dist": 36530,
"calc_other": 87223,
"calc_total": 279752
}
]
];
var uniqExp = _.uniq(_.pluck(flattened, 'expense'));
$scope.exp = uniqExp;
});
} // end BudgetsController fn
})();
Let's see what you've got:
_.groupBy: returns an object whose keys are crop's names;
_.map: iterates over grped keys (crop names) and returns an array;
item.reduce(): native array method, accumulates some values through the whole array
current: reduces passes a different element from array, each time;
previous: this one we're are in control, the first time it contains the value of the second reduce() parameter, this is why I've passed an fixed object. This object is modified each iteration.
More information about Array.prototype.reduce: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array/Reduce
Sample:
$scope.uses = _.map(grped, function (item, key) {
return item.reduce(function (previous, current) {
// in the nth iteration, each previous property will be equal to sum(arr[0].property...arr[n-1].property)
previous.arm += current.arm;
previous.dist += current.dist;
previous.other += current.other;
// other fields should be summed here
// remember to return the accumulator
return previous;
},
/* initialize each property to zero, otherwize it won't work */
{crop: key, arm: 0, dist: 0, other: 0});
});

Initialize a multdimensional array in C similar to Fortran

I've written a small test case program in FORTRAN 90 which initializes a 3d array in slices. Can this same behavior be easily replicated in C?
program fortranEngine
real(4) , dimension(10,10) :: tmp
real(4) , dimension(10,10,3) :: p
tmp = RESHAPE( [ 0.973, 1.053, 0, 0, 0, 0, 0, 0, 0, 0, &
1.053, 1.080, 0, 0, 0, 0, 0, 0, 0, 0, &
1.010, 0.408, 0.442, 0, 0, 0, 0, 0, 0, 0, &
1.119, 0.900, 0.399, 0.762, 0, 0, 0, 0, 0, 0, &
1.211, 0.975, 0.845, 0.952, 1.105, 0, 0, 0, 0, 0, &
1.248, 1.016, 0.485, 0.000, 0.000, 1.110, 0, 0, 0, 0, &
1.225, 1.123, 1.056, 0.000, 0.000, 0.949, 0.832, 0, 0, 0, &
1.138, 1.232, 1.089, 1.050, 0.930, 0.402, 0.789, 0.774, 0, 0, &
1.149, 1.406, 1.201, 1.052, 0.416, 0.878, 0.896, 0.431, 1.144, 0, &
1.351, 1.255, 1.290, 1.358, 1.240, 1.228, 1.257, 1.140, 1.123, 1.228] &
, [10,10] )
p(:,:,1) = tmp
...
end program fortranEngine
You can more or less do it in C99 or C2011, but it isn't as convenient as in Fortran. Beware initialization order; Fortran does it in one order (column-major) and C does it in the other (row-major). Ignoring that, you can do:
float tmp[10][10] =
{
{ 0.973, 1.053, 0, 0, 0, 0, 0, 0, 0, 0, },
{ 1.053, 1.080, 0, 0, 0, 0, 0, 0, 0, 0, },
{ 1.010, 0.408, 0.442, 0, 0, 0, 0, 0, 0, 0, },
{ 1.119, 0.900, 0.399, 0.762, 0, 0, 0, 0, 0, 0, },
{ 1.211, 0.975, 0.845, 0.952, 1.105, 0, 0, 0, 0, 0, },
{ 1.248, 1.016, 0.485, 0.000, 0.000, 1.110, 0, 0, 0, 0, },
{ 1.225, 1.123, 1.056, 0.000, 0.000, 0.949, 0.832, 0, 0, 0, },
{ 1.138, 1.232, 1.089, 1.050, 0.930, 0.402, 0.789, 0.774, 0, 0, },
{ 1.149, 1.406, 1.201, 1.052, 0.416, 0.878, 0.896, 0.431, 1.144, 0, },
{ 1.351, 1.255, 1.290, 1.358, 1.240, 1.228, 1.257, 1.140, 1.123, 1.228 },
};
float p[3][10][10];
for (int i = 0; i < 3; i++)
memmove(p[i], tmp, sizeof(tmp));
Note that I moved the dimension [3] from the end of the declaration to the start of the declaration, though. The other way around would not make much sense in C. So, the notation is more or less available, but details of storage management make it less than obvious how to achieve exactly what you want.
Is this easy enough?
for(int i = 0 ; i < 10 ; ++i)
for(int j = 0 ; j < 10 ; ++j )
p[0][i][j] = tmp[i][j] ;

Resources