Create Numpy Grid in correct order - arrays

I have several arrays with points along my x-, y-,...-axes and want to create a grid with all points in it like this:
x = [1, 2, 3]
y = [20, 40, 60, 80]
result = []
for xi in x:
for yi in y:
result.append([xi, yi])
np.array(result)
which gives me
array([[ 1, 20],
[ 1, 40],
[ 1, 60],
[ 1, 80],
[ 2, 20],
[ 2, 40],
[ 2, 60],
[ 2, 80],
[ 3, 20],
[ 3, 40],
[ 3, 60],
[ 3, 80]])
To do this with numpy I found the following code in this question:
np.vstack(np.meshgrid(x, y)).reshape(2, -1).T
But this gives the result in the wrong order:
array([[ 1, 20],
[ 2, 20],
[ 3, 20],
[ 1, 40],
[ 2, 40],
[ 3, 40],
[ 1, 60],
[ 2, 60],
[ 3, 60],
[ 1, 80],
[ 2, 80],
[ 3, 80]])
It goes through the x-values first, then y-values.
I can get around this by using
np.vstack(reversed(np.meshgrid(y, x))).reshape(2, -1).T
but this does not work any more in 3D, where
np.vstack(np.meshgrid(x, y, z)).reshape(3, -1).T
goes through z-values first, then x-values, then y-values.
How can I get the correct order in all dimensions with numpy?

You can specify the matrix indexing ij in np.meshgrid as the indexing parameter to get the reversed order, by default it's the cartesian indexing order xy:
x = [1, 2, 3]
y = [20, 40, 60, 80]
np.stack(np.meshgrid(x, y, indexing='ij'), axis=-1).reshape(-1, 2)
#array([[ 1, 20],
# [ 1, 40],
# [ 1, 60],
# [ 1, 80],
# [ 2, 20],
# [ 2, 40],
# [ 2, 60],
# [ 2, 80],
# [ 3, 20],
# [ 3, 40],
# [ 3, 60],
# [ 3, 80]])
In 3d this could be:
np.stack(np.meshgrid(x, y, z, indexing='ij'), axis=-1).reshape(-1, 3)

Related

How to create a 3D list of 2D lists that vary in size(The 2D lists vary in size) in Python?

I have my dataframe like this. You can see that the column called filename indicates that this row is in a file and another row is in a different file or not. I have also created another column to count the total number of rows are in a file.
I have extracted the ymin and ymax by concatenating all of them to a list and the result is a 2D list:
y = [[4, 43], [9, 47], [76, 122], [30, 74], [10, 47], [81, 125], [84, 124], [47, 90], [1, 38],[2, 40], [2, 44], [4, 48], [5, 48], [6, 44], [8, 45], [75, 116], [73, 123], [28, 73], [39, 84], [84, 121], [2, 39],...]
Thus, this only puts all the coordinates into a list without knowing which belongs to the first file and which belongs to the second
My approach is making a 3D list like this:
y = [[[4, 43], [9, 47], [76, 122], [30, 74], [10, 47], [81, 125], [84, 124], [47, 90], [1, 38]],[[2, 40], [2, 44], [4, 48], [5, 48], [6, 44], [8, 45], [75, 116], [73, 123], [28, 73], [39, 84], [84, 121], [2, 39]],...]
You can see from [4,43] to [1,38] that they are in the same file.
You can also see from [2,40] to [2,39] that they are also in the same file.
Here is my current attempt
def get_y_coordinate(count):
"""
Create a 3D list of y_coordinates that can distinguish which list is in a file which list belongs to another file
:param: count - the list taken from the column "count" from the dataframe
"""
c = 2 # Number of chunks to make
fi_y= lambda y, c: [y[i:i+c] for i in range(0, len(y), c)] # Making y into chunks of 2 ex: from [4,43,9,47] to [4,43],[9,74]
y = fi_y(y,c) # Now y is [[4, 43], [9, 47],...]]
# This is my current approach, I create a new list called bigy.
bigy = []
current = 0
for i in count:
if current != i:
bigy.append([y[j] for j in range(current, i+current)])
current = i
return bigy
>> bigy = [[[4, 43], [9, 47], [76, 122], [30, 74], [10, 47], [81, 125], [84, 124], [47, 90], [1, 38]],[[2, 40], [2, 44], [4, 48], [5, 48], [6, 44], [8, 45], [75, 116], [73, 123], [28, 73], [39, 84], [84, 121], [2, 39]],...]
I achieved the result for a first few hundreds of files. However, up until around file 700, it doesn't work anymore. I need another approach for this problem if anyone can be patient enough to read to here and help me out. Thank you very much!
I think my first inclination would be to just iterate over the dataframe and collect results in a defaultdict. Maybe something like:
import collections
import pandas
mock_data = pandas.DataFrame([
{"Name": "product_name", "ymin": 4, "ymax": 43},
{"Name": "product_name", "ymin": 9, "ymax": 47},
{"Name": "product_total_money", "ymin": 76, "ymax": 122},
{"Name": "vat", "ymin": 30, "ymax": 74},
{"Name": "product_name", "ymin": 10, "ymax": 47}
])
y_results = collections.defaultdict(list)
for _, row in mock_data.iterrows():
y_results[row["Name"]].append((row["ymin"], row["ymax"]))
print(y_results)
Alternately, you might also try:
mock_data.groupby('Name').agg(lambda x: list(x))

masking a multidimensional array - numpy

I have two identical arrays of size (2,3,2)
[[[1, 7],
[2, 8],
[3, 9]],
[[4, 10],
[5, 11],
[6, 12]]]
stored in a multidimensional array
>>> a = np.array([[[[ 1, 1],
[ 7, 7]],
[[ 2, 2],
[ 8, 8]],
[[ 3, 3],
[ 9, 9]]],
[[[ 4, 4],
[10, 10]],
[[ 5, 5],
[11, 11]],
[[ 6, 6],
[12, 12]]]])
>>> a.shape
(2, 3, 2, 2)
I'm trying to mask the sub arrays in a with m:
>>> m = np.array([[[1, 0],
[0, 0],
[1, 1]],
[[1, 1],
[0, 1],
[0, 0]]])
which should result in:
[[[[ 1, 1],
[ 0, 0]
[ 0, 0],
[ 0, 0],
[ 3, 3],
[ 9, 9]]
[[ 4, 4],
[10, 10],
[ 0, 0],
[11, 11],
[ 0, 0],
[ 0, 0]]]
I tried to use np.concatenate and np.append eg, np.prod(np.concatenate([a,m],axis=0))
but none of my solutions worked.
Expand the dimensions of m so it broadcasts with a:
In [183]: a*m[...,None]
Out[183]:
array([[[[ 1, 1],
[ 0, 0]],
[[ 0, 0],
[ 0, 0]],
[[ 3, 3],
[ 9, 9]]],
[[[ 4, 4],
[10, 10]],
[[ 0, 0],
[11, 11]],
[[ 0, 0],
[ 0, 0]]]])

How to subtract arrays of integers

I have four arrays of int:
num_defect = [30, 30, 20, 20, 18, 18, 5, 5]
num_fixes = [1, 0, 3, 2, 1, 2, 2, 2]
num_blocks = [0, 0, 0, 0, 2, 2, 1, 0]
num_ext_defects = [1, 1, 0, 0, 2, 2, 2, 1]
I want to display the number of open defects, which is given by:
num_defects - num_fixes - num_blocks - num_ext_defects
So for the reporting, num_defects should now contain:
[28, 29, 17, 13, 12, 0, 2]
I tried:
num_defect.map { |i| i - num_fixes[i] - num_blocks[i] - num_ext_defects[i] }
but it raises:
nil can't be coerced into Fixnum
Any help greatly appreciated.
With
num_defect.map { |i|
i is the element of the array, not its index. If you want your map to work correctly, you'll need an index as well:
num_defect.map.with_index do |element, index|
element - num_fixes[index] - num_blocks[index] - num_ext_defects[index]
end
Use map! instead of map in order to mutate num_defect.
Or if you'd like a nicer version:
a = [30,30,20,20,18,18,5,5]
b = [ 1, 0, 3, 2, 1, 2,2,2]
c = [ 0, 0, 0, 0, 2, 2,1,0]
d = [ 1, 1, 0, 0, 2, 2,2,1]
a.zip(b,c,d).map { |arr| arr.inject(:-) }
# => [28, 29, 17, 18, 13, 12, 0, 2]
If i understand you correctly you might be looking for an array method called each_index.
num_defect.each_index do |i|
num_defect[i] -= num_fixes[i] + num_blocks[i] + num_ext_defects[i]
end
require 'matrix'
(Vector.elements(num_defect) - Vector.elements(num_fixes) -
Vector.elements(num_blocks) - Vector.elements(num_ext_defects)).to_a
#=> [28, 29, 17, 18, 13, 12, 0, 2]
This uses the methods Vector::elements and Vector#to_a. One could write Vector[*arr] in place of Vector.elements(arr), using Vector::[].
If num_defect is to be mutated, you could write num_defect.replace(<above expression>). If
arr = [num_defect, num_fixes, num_blocks, num_ext_defects]
#=> [[30, 30, 20, 20, 18, 18, 5, 5],
# [ 1, 0, 3, 2, 1, 2, 2, 2],
# [ 0, 0, 0, 0, 2, 2, 1, 0],
# [ 1, 1, 0, 0, 2, 2, 2, 1]]
one could use matrix multiplication:
(Matrix.row_vector([1, *[-1]*(arr.size-1)]) * Matrix.rows(arr)).to_a.first
#=> [28, 29, 17, 18, 13, 12, 0, 2]
where
[1, *[-1]*(arr.size-1)]
#=> [1, -1, -1, -1]
This would be convenient and relatively computationally-efficient if arr had a larger number of elements than it does in the example.
This uses the Matrix methods Matrix::row_vector, Matrix::rows and Matrix#to_a. One could write Matrix[*arr] in place of Matrix.rows(arr), using Matrix::[]. One advantage of using rows, however, is that one can add the argument false (Matrix.rows(arr, false)) to avoid copying the elements of arr in the creation of the Matrix object.
[num_defect, num_fixes, num_blocks, num_ext_defects]
.transpose
.map{|first, *rest| first - rest.sum}
# => [28, 29, 17, 18, 13, 12, 0, 2]
Using Enumerator#each_with_object:
num_defect.each_with_index.with_object([]){ |(e, i), a| a << (e - num_fixes[i] - num_blocks[i] - num_ext_defects[i]) }
#=> [28, 29, 17, 18, 13, 12, 0, 2]

How to change value of an element in dictionary of arrays?

I have created a dictionary like this
var MyArray: [String:[String:[Int]]] = [
"xx": ["x1": [1, 2, 3], "x2": [4, 5, 6], "x3": [7, 8, 9]],
"yy": ["y1": [10, 11, 12], "y2": [13, 14, 15], "y3": [16, 17, 18]]]
How can I change the value 3 in "x1" in "xx" to an other number?
I don't know that this is the number 3 but i know that it is in MyArray["xx"]!["x1"]![2]
// example setup
var myArray: [String:[String:[Int]]] = [
"xx": ["x1": [1, 2, 3], "x2": [4, 5, 6], "x3": [7, 8, 9]],
"yy": ["y1": [10, 11, 12], "y2": [13, 14, 15], "y3": [16, 17, 18]]]
// value to be replaced
let oldNum = 3
// value to replace old value by
let newNum = 4
// extract the current value (array) for inner key 'x1' (if it exists),
// and proceed if 'oldNum' is an element of this array
if var innerArr = myArray["xx"]?["x1"], let idx = innerArr.index(of: oldNum) {
// replace the 'oldNum' element with your new value in the copy of
// the inner array
innerArr[idx] = newNum
// replace the inner array with the new mutated array
myArray["xx"]?["x1"] = innerArr
}
print(myArray)
/* ["yy": ["y3": [16, 17, 18], "y2": [13, 14, 15], "y1": [10, 11, 12]],
"xx": ["x1": [1, 2, 4], "x3": [7, 8, 9], "x2": [4, 5, 6]]]
^ ok! */
Based on the following Q&A:
Dictionary in Swift with Mutable Array as value is performing very slow? How to optimize or construct properly?
A more performant approach would be actually removing the inner array (for key "x1"); mutating it; and re-adding it to the dictionary
// check if 'oldNum' is a member of the inner array, and if it is: remove
// the array and mutate it's 'oldNum' member to a new value, prior to
// adding the array again to the dictionary
if let idx = myArray["xx"]?["x1"]?.index(of: oldNum),
var innerArr = myArray["xx"]?.removeValue(forKey: "x1") {
innerArr[idx] = newNum
myArray["xx"]?["x1"] = innerArr
}
print(myArray)
// ["yy": ["y3": [16, 17, 18], "y2": [13, 14, 15], "y1": [10, 11, 12]], "xx": ["x1": [1, 2, 4], "x3": [7, 8, 9], "x2": [4, 5, 6]]]
If you know the index of the number you would like to change out of the three, you can change the number 3 directly using the subscripts ["xx"]?["x1"]?[2].
var myArray = [
"xx": [
"x1": [1, 2, 3],
"x2": [4, 5, 6],
"x3": [7, 8, 9]
],
"yy": [
"y1": [10, 11, 12],
"y2": [13, 14, 15],
"y3": [16, 17, 18]
]
]
array["xx"]?["x1"]?[2] = 4

AutoIT array missing bracket

I have this array:
Global Const $array[70][8] = [[1, ...], [2, ...], [3, ...], ... , [70, 0.48124164, 0.88451159, -8855, -612, 176, 1, 7]]
If i run the code, it says "Error: Missing right bracked ')' in expression". If i remove the last element it works. I can't get why the last element is producing this error, and i can't find any syntax error.
I also can compile or build it without any error.
Has anyone an idea what can cause this behaviour?
//EDIT
Here is the full array definition:
Global Const $array[70][8] = [[1, 0.46382308, 0.37846267223358, 3476, 20, 4, -1, 4], [2, 0.66173166, 0.35245704650879, 4268, -7333, 7, 3, 4], [3, 0.43819791, 0.73096454143524, -5063, 891, 158, 2, 4], [4, 0.49637758, 0.27708977460861, 6028, -1121, 381, 1, 4], [5, 0.60390532, 0.37993001937866, 3503, -5119, 83, 2, 4], [6, 0.43493705, 0.50274324417114, 415, 1054, 130, 3, 4], [7, 0.5436604, 0.88573443889618, -9015, -2928, 52, 1, 3], [8, 0.51811683, 0.60228180885315, -1968, -2021, 95, 1, 3], [9, 0.55463874, 0.84207928180695, -7663, -3285, 70, 1, 3], [10, 0.48325252, 0.79658991098404, -6724, -746, -272, 1, 3], [11, 0.5183885, 0.8305846452713, -7633, -1877, -272, -1, 3], [12, 0.55923116, 0.68494528532028, -3946, -3505, 36, 2, 3], [13, 0.48257315, 0.8402858376503, -7837, -644, -262, 2, 3], [14, 0.55860614, 0.70989108085632, -4713, -3447, 36, 2, 3], [15, 0.42719244, 0.61063778400421, -2193, 1338, 70, 2, 3], [16, 0.41371411, 0.57615393400192, -1314, 1882, 48, 2, 3], [17, 0.50219285, 0.79606002569199, -6684, -1330, -272, 2, 3], [18, 0.47042638, 0.43658798933029, 2099, -196, 97, 2, 4], [19, 0.42469245, 0.41934603452682, 2529, 1427, 265, 3, 4], [20, 0.43958377, 0.52923792600632, -262, 827, 92, 2, 4], [21, 0.49751889, 0.78456538915634, -6454, -1299, -268, 2, 3], [22, 0.48034489, 0.50571876764297, 363, -592, 21, 4, 3], [23, 0.39637708, 0.53408849239349, -372, 2397, 39, 3, 4], [24, 0.39023572, 0.64112710952759, -2920, 2726, 75, 2, 4], [25, 0.41588801, 0.64752662181854, -3116, 1715, 51, 2, 1], [26, 0.45412194, 0.81338351964951, -7201, 402, 24, 3, 4], [27, 0.50868737, 0.36688649654388, 3793, -1674, 248, 2, 4], [28, 0.5741769, 0.25597554445267, 6584, -4035, 661, 4, 4], [29, 0.52428531, 0.59066486358643, -1700, -2216, 97, 2, 4], [30, 0.43132287, 0.72403514385223, -5038, 1220, 51, 1, 4], [31, 0.51977443, 0.34100317955017, 4459, -2070, 1206, 2, 4], [32, 0.4120565, 0.59812414646149, -1975, 1930, 62, 3, 4], [33, 0.59803569, 0.29261976480484, 5669, -4950, 805, 3, 4], [34, 0.4343664, 0.67442893981934, -3846, 1152, 154, 5, 4], [35, 0.46480131, 0.91842484474182, -9804, -73, 67, 3, 7], [36, 0.56852471, 0.84611463546753, -7977, -3894, 9, 1, 8], [37, 0.47779053, 0.32180470228195, 4813, -401, 349, 2, 4], [38, 0.52241027, 0.30594861507416, 5344, -2175, 1278, 2, 4], [39, 0.52458423, 0.6877578496933, -4022, -2207, 94, 3, 1], [40, 0.55643224, 0.88345181941986, -8904, -3372, 16, -1, 8], [41, 0.43931204, 0.35995709896088, 3888, 772, 5, 3, 4], [42, 0.54311692, 0.80111443996429, -6882, -2920, 22, -1, 8], [43, 0.42798048, 0.70581495761871, -4682, 1328, 94, -1, 4], [44, 0.3859694, 0.54077327251434, -486, 2956, 63, -1, 4], [45, 0.47751879, 0.33729392290115, 4491, -594, 283, -1, 4], [46, 0.43974685, 0.33378851413727, 4621, 945, 53, 1, 4], [47, 0.51083415, 0.31422311067581, 5179, -1736, 1338, 3, 4], [48, 0.45425778, 0.89918559789658, -9241, 309, 262, 3, 7], [49, 0.36626821, 0.71910309791565, -4933, 3595, 10, 3, 4], [50, 0.550481, 0.33325856924057, 4563, -2943, 1062, 4, 4], [51, 0.45588821, 0.8790088891983, -8811, 306.7, 346.9, 3, 7], [52, 0.54849743, 0.403449177742, 2900, -3145, 177, 3, 4], [53, 0.59265524, 0.30664157867432, 5234, -4822, 691, 3, 4], [54, 0.42800766, 0.74604618549347, -5443, 1265, 20, 3, 4], [55, 0.4743666, 0.874076783, -8673, -364, 191, 1, 7], [56, 0.45151323, 0.957066357, -10774, 383, 25, 3, 7], [57, 0.47893184, 0.984906196, -11285, -681, 121, 4, 7], [58, 0.50632327, 0.886182785, -8951, -1578, 94, 3, 7], [59, 0.52273643, 0.937378764, -10108, -2216, 49, 4, 7], [60, 0.52306246, 0.924375951, -9920, -2244, 27, 4, 7], [61, 0.54613327, 0.291233837, 5638, -3031, 1559, 1, 4], [62, 0.54523652, 0.89673995, -9217, -3016, 20, -1, 8], [63, 0.50667655, 0.897432863, -9184, -1735, 73, 2, 7], [64, 0.53928542, 0.9322021, -10176, -2685, 5, 2, 7], [65, 0.5081439, 0.97875124, -11280, -1734, 0, 1, 7], [66, 0.45591539, 0.87778604, -8811, 315, 348, 4, 7], [67, 0.53086143, 0.92669934, -9956, -2405, 37, 2, 7], [68, 0.49053514, 0.938356995, -10244, -1009, 47, 1, 7], [69, 0.43232834, 0.950096189, -10584, 1205, 31, 1, 7], [70, 0.48124164, 0.88451159, -8855, -612, 176, 1, 7]]
The code around looks like
EndFunc
Global Const first[54][8] = ...
Global Const $second[29][8] = ...
Global Const $array[70][8] = [...] // error array
Global Const $last[30][8] = ...
Func function($first)
Look like a fat finger:
Local $arr[3][3] = [[1, 2, 3], [2, 3, 4], [3, (.00, 0.1575818700]]
==> Missing right bracket ')' in expression.:

Resources