I am generating two parameters e.g.
s1 = [0, 0.25, 0.5, 0.75, 1.0]
s2 = [0, 0.25, 0.5, 0.75, 1.0]
based on dimensions of both these lists above, i am creating a grid of zeros:
np.zeros((5,5))
I then pair up each of the numbers in each list so they form coordinate locations in my empty grid e.g. (0,0), (0,0.25), (0,0.5) etc. (25 combinations to fit into 5x5 grid).
my issue is i am not too sure how to append values into the grid based on each of the coordinates generated. e.g. if i want to append the number 5 to grid location (0,0) etc so the grid fills up.
Any help is greatly appreciated.
The easiest way is probably to construct a meshgrid and transpose it so that the axises are the way you want them:
np.array(np.meshgrid(s1, s2)).transpose(1, 2, 0)
not sure it is fastest way to get it, check if the output is what you were expecting
import numpy as np
s1 = [0, 0.25, 0.5, 0.75, 1.0]
s2 = [0, 0.25, 0.5, 0.75, 1.0]
arr = np.zeros((5,5,2), dtype=float)
print (arr.shape, arr.size, arr.ndim)
for i in range(len(s1)):
for j in range(len(s2)):
arr[i,j] = s1[i], s2[j]
print(arr)
output :
(5, 5, 2) 50 3
[[[0. 0. ]
[0. 0.25]
[0. 0.5 ]
[0. 0.75]
[0. 1. ]]
[[0.25 0. ]
[0.25 0.25]
[0.25 0.5 ]
[0.25 0.75]
[0.25 1. ]]
[[0.5 0. ]
[0.5 0.25]
[0.5 0.5 ]
[0.5 0.75]
[0.5 1. ]]
[[0.75 0. ]
[0.75 0.25]
[0.75 0.5 ]
[0.75 0.75]
[0.75 1. ]]
[[1. 0. ]
[1. 0.25]
[1. 0.5 ]
[1. 0.75]
[1. 1. ]]]
Related
I have an array of sorted numbers:
arr = [-0.1, 0.0, 0.5, 0.8, 1.2]
I want the difference (dist below) between consecutive numbers for that array to be above a given threshold. For example, if threshold is 0.25:
dist = [0.1, 0.5, 0.3, 0.4] # must be >0.25 for all elements
arr[0] and arr[1] are too close to each other, so one of them must be modified. In this case the desired array would be:
good_array = [-0.25, 0.0, 0.5, 0.8, 1.2] # all elements distance > threshold
In order to obtain good_array, I want to modify the minimum amount of elements in arr. So I substract 0.15 from arr[0] rather than, say, substract 0.1 from arr[0] and add 0.05 to arr[1]:
[-0.2, 0.05, 0.5, 0.8, 1.2]
Previous array is also valid, but we have modified 2 elements rather than one.
Also, in case it is possible to generate good_array by modifying different elements in arr, by default modify the element closer to the edge of the array. But keep in mind the main goal is to generate good_array by modifying the minimum number of elemtns in arr.
[-0.1, 0.15, 0.5, 0.8, 1.2]
Previous array is also valid, but we have modified arr[1] rather than the element closer to the edge (arr[0]). In case 2 elements have equal distance from edges, modify the one closer to begining of array:
[-0.3, 0.15, 0.2, 0.7] # modify arr[1] rather than arr[2]
So far I have been doing this manually for small arrays, but I would like a general solution for larger arrays.
Here is brute force python solution, where we try to fix elements to the right or elements to the left when there is a conflict:
def solve(arr, thereshold):
original = list(arr)
def solve(idx):
if idx + 1 >= len(arr):
return [sum(1 for x in range(len(arr)) if arr[x] != original[x]), list(arr)];
if arr[idx + 1] - arr[idx] < thereshold:
copy = list(arr)
leftCost = 0
while idx - leftCost >= 0 and arr[idx + 1] - arr[idx - leftCost] < thereshold * (leftCost + 1):
arr[idx - leftCost] = arr[idx - leftCost + 1] - thereshold
leftCost += 1
left = solve(idx + 1)
for cost in range(leftCost):
arr[idx - cost] = copy[idx - cost]
rightCost = 0
while idx + rightCost + 1 < len(arr) and arr[idx + rightCost + 1] - arr[idx] < thereshold * (rightCost + 1):
arr[idx + rightCost + 1] = arr[idx + rightCost ] + thereshold
rightCost += 1
right = solve(idx + 1)
for cost in range(rightCost):
arr[idx + cost + 1] = copy[idx + cost + 1]
if right[0] < left[0]:
return right
elif left[0] < right[0]:
return left
else:
return left if idx - left[0] <= len(arr) - idx - right[0] else right
else:
return solve(idx + 1)
return solve(0)
print(solve([0,0.26,0.63,0.7,1.2], 0.25))
Edit: I just realized that my original solution was stupid and overcomplicated. Now presenting simple and better solution
First approach
If I understand your problem correctly, your input array can have some regions, where your condition is not met. For instance:
array = [0.0, 0.0, 0.0, 0.0, 0.0, 0.25, 0.5, 0.75, 1.0] (first 4 elements)
or:
array = [0.25, 0.5, 0.75, 1.0, 1.0, 1.0, 1.0, 1.0, 1.25, 1.5, 1.75] (elements arr[4], arr[5] and arr[6])
To fix that, you have to add (or subtract) some pattern like:
fixup = [0.0, 0.25, 0.0, 0.25, 0.0, 0.0, 0.0, 0.0, 0.0] (for the first case)
or:
fixup = [0.0, 0.0, 0.0, 0.0, 0.25, 0.0, 0.25, 0.0, 0.0, 0.0, 0.0] (for the second example)
Second approach
But our current solution has got some problem. Consider a bad area with an "elevation":
array = [0.0, 0.25, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 1.1, 1.35, 1.6] (broken area is within values: 0.6-1.0)
In that case our correct "solution" will be:
fixup = [0.0, 0.0, 0.0, 0.25+0.1, 0.0, 0.25+0.1, 0.0, 0.25+0.1, 0.0, 0.0, 0.0]
which produce:
good_array = [0.0, 0.25, 0.5, 0.95, 0.7, 1.15, 0.9, 1.0, 1.1, 1.35, 1.6]
So to summarize, you have to apply the "patch":
fixup[i] = threshold+max(difference[i], difference[i-1]) (for i when i-start_index is even)
(please note that it will be -threshold+min(difference[i], difference[i-1]) for negative values)
and:
fixup[i] = 0 (for i when i-start_index is odd)
start_index is a beginning of the bad region.
Third approach
Previously mentioned formula doesn't work well for some cases (like [0.1, 0.3, 0.4] that it would increment 0.3 up to 0.75 when only 0.65 is sufficient)
Lets try to improve that:
good_array[i] = max(threshold+array[i-1], threshold+array[i+1]) (for abs(array[i-1]-array[i+1]) < threshold*2)
and:
good_array[i] = (array[i-1]+array[i+1])/2 otherwise.
(you can also choose formula: good_array[i] = min(-threshold+array[i-1], -threshold+array[i+1]) when it would produce a result closer to original array value, if minimizing difference is also your optimization goal)
4th approach
Bad regions of even length are also a threat. I can think about 2 ways to solve it:
Solution based on a pattern like [0.0, 0.25, 0.5, 0.0]
Or based on a pattern like [0.0, 0.25, -0.25, 0.0] (We are simply using "the second formula")
Or [0.0, 0.25, 0.0, 0.25] (just including additional element to make bad region length odd -I don't recommend this approach as it would require handling lot of corner cases)
Corner cases
Please consider also some corner cases (bad region starts or ends at an "edge" of the array):
good_array[0] = threshold+array[1]
and:
good_array[array_size-1] = threshold+array[array_size-2]
Final hints
I would suggest to implement lot of unit tests during implementation in order to easily verify correctness of derived formulas and handle some combinations of corner cases. Bad areas that consist of only one element can be one of them.
I'm working with a 3d array of vectors, and having trouble reshaping properly.
My dimensions correspond to quantities as follows:
0 = vector (3)
1 = point (4)
2 = polyline (2)
So this can be interpreted as 2 polylines that each contain 4 points, and each point has a vector. I want to reshape to a 2d matrix that is (3, 8).
The original array is:
poly_array = array([[[-0.707, 0.0],
[-0.371, 0.0],
[0.371, 0.0],
[0.707, 0.0]],
[[0.0, -0.707],
[0.0, 0.0],
[0.0, 0.707],
[0.0, 0.0]],
[[0.707, 0.707],
[0.928, 1.0],
[0.928, 0.707],
[0.707, 0.0]]])
so if I'm looking at ordered points along the first polyline, I would run:
for i in range(4):
print poly_array[:,i,0]
or for ordered points along the second polyline:
for i in range(4):
print poly_array[:,i,1]
If I reshape this way:
new_dim = shape(poly_array)[1] * shape(poly_array)[2]
new_array = poly_array.reshape(3, new_dim)
But this orders the vectors as taking one from each polyline (i.e., pt0-polyline0, pt0-polyline1, pt1-polyline0, pt1-polyline1, etc.)
In: print new_array[:, 0]
Out: [-0.707 0. 0.707]
In: print new_array[:, 1]
Out: [ 0. -0.707 0.707]
But I want
In: print new_array[:, 1]
Out: [-0.371 0. 0.928]
How can I reshape so that it loops through all the vectors corresponding to points (along axis 1) for a given polyline before the next polyline?
You would need some permuting of axes with np.swapaxes and a reshape -
poly_array.swapaxes(1,2).reshape(poly_array.shape[0],-1)
Sample run -
In [127]: poly_array
Out[127]:
array([[[-0.707, 0. ],
[-0.371, 0. ],
[ 0.371, 0. ],
[ 0.707, 0. ]],
[[ 0. , -0.707],
[ 0. , 0. ],
[ 0. , 0.707],
[ 0. , 0. ]],
[[ 0.707, 0.707],
[ 0.928, 1. ],
[ 0.928, 0.707],
[ 0.707, 0. ]]])
In [142]: out
Out[142]:
array([[-0.707, -0.371, 0.371, 0.707, 0. , 0. , 0. , 0. ],
[ 0. , 0. , 0. , 0. , -0.707, 0. , 0.707, 0. ],
[ 0.707, 0.928, 0.928, 0.707, 0.707, 1. , 0.707, 0. ]])
In [143]: out[:,1]
Out[143]: array([-0.371, 0. , 0.928])
I'm trying to index some matrix, y, and then reindex that result with some boolean statement and set the corresponding elements in y to 0. The dummy code I'm using to test this indexing scheme is shown below.
x=np.zeros([5,4])+0.1;
y=x;
print(x)
m=np.array([0,2,3]);
y[0:4,m][y[0:4,m]<0.5]=0;
print(y)
I'm not sure why it does not work. The output I want:
[[ 0.1 0.1 0.1 0.1]
[ 0.1 0.1 0.1 0.1]
[ 0.1 0.1 0.1 0.1]
[ 0.1 0.1 0.1 0.1]
[ 0.1 0.1 0.1 0.1]]
[[ 0. 0.1 0. 0. ]
[ 0. 0.1 0. 0. ]
[ 0. 0.1 0. 0. ]
[ 0. 0.1 0. 0. ]
[ 0.1 0.1 0.1 0.1]]
But what I actually get:
[[ 0.1 0.1 0.1 0.1]
[ 0.1 0.1 0.1 0.1]
[ 0.1 0.1 0.1 0.1]
[ 0.1 0.1 0.1 0.1]
[ 0.1 0.1 0.1 0.1]]
[[ 0.1 0.1 0.1 0.1]
[ 0.1 0.1 0.1 0.1]
[ 0.1 0.1 0.1 0.1]
[ 0.1 0.1 0.1 0.1]
[ 0.1 0.1 0.1 0.1]]
I'm sure I'm missing some under-the-hood details that explains why this does not work. Interestingly, if you replace m with :, then the assignment works. For some reason, selecting a subset of the columns does not let me assign the zeros.
If someone could explain what's going on and help me find an alternative solution (hopefully one that does not involve generating a temporary numpy array since my actual y will be really huge), I would really appreciate it! Thank you!
EDIT:
y[0:4,:][y[0:4,:]<0.5]=0;
y[0:4,0:3][y[0:4,0:3]<0.5]=0;
etc.
all work as expected. It seems the issue is when you index with a list of some kind.
Make an array (this is one of my favorites because the values differ):
In [845]: x=np.arange(12).reshape(3,4)
In [846]: x
Out[846]:
array([[ 0, 1, 2, 3],
[ 4, 5, 6, 7],
[ 8, 9, 10, 11]])
In [847]: m=np.array([0,2,3])
In [848]: x[:,m]
Out[848]:
array([[ 0, 2, 3],
[ 4, 6, 7],
[ 8, 10, 11]])
In [849]: x[:,m][:2,:]=0
In [850]: x
Out[850]:
array([[ 0, 1, 2, 3],
[ 4, 5, 6, 7],
[ 8, 9, 10, 11]])
No change. But if I do the indexing in one step, it changes.
In [851]: x[:2,m]=0
In [852]: x
Out[852]:
array([[ 0, 1, 0, 0],
[ 0, 5, 0, 0],
[ 8, 9, 10, 11]])
it also works if I reverse the order:
In [853]: x[:2,:][:,m]=10
In [854]: x
Out[854]:
array([[10, 1, 10, 10],
[10, 5, 10, 10],
[ 8, 9, 10, 11]])
x[i,j] is executed as x.__getitem__((i,j)). x[i,j]=v as x.__setitem__((i,j),v).
x[i,j][k,l]=v is x.__getitem__((i,j)).__setitem__((k,l),v).
The set applies to the value produced by the get. If the get returns a view, then the change affects x. But if it produces a copy, the change does not affect x.
With array m, y[0:4,m] produces a copy (do I need to demonstrate that?). y[0:4,:] produces a view.
So in short, if the first indexing produces a view the second indexed assignment works. But if produces a copy, the second has no effect.
import numpy as np
X_mini=np.array([[ 4, 2104, 1],
[ 1, 1600, 3],
[ 3, 2400, 100]])
def feature_normalization(X):
row_length=len(X[0:1][0])
for i in range(0, row_length):
if not X[:,i].std()==0:
temp=(X[:,i]-X[:,i].mean())/X[:,i].std()
print(temp)
X[:,i]=temp
feature_normalization(X_mini)
print(X_mini)
outputs:
[ 1.06904497 -1.33630621 0.26726124]
[ 0.209937 -1.31614348 1.10620649]
[-0.72863911 -0.68535362 1.41399274]
[[ 1 0 0]
[-1 -1 0]
[ 0 1 1]]
my question is, why does not X_mini (after applying feature_normalization) correspond to what is being printed out?
Your array holds values of integer type (probably int64).
When fractions are inserted into it, they're converted to int.
You can explicitly specify the type of an array you create:
X_mini = np.array([[ 4.0, 2104.0, 1.0],
[ 1.0, 1600.0, 3.0],
[ 3.0, 2400.0, 100.0]], dtype=np.float128)
You can also convert an array to another type using numpy.ndarray.astype (docs).
I am working in python. I have an angle quantity for which I want a varying step size for the array instead of a uniform grid that can be created like np.linspace(0, pi, 100) for 100 equal steps. Instead, I want more 'resolution' (i.e. a smaller step-size) for values close to 0 and pi, with larger step sizes closer to pi/2 radians. Is there a simple way to implement this in python using a technique already provided in numpy or otherwise?
Here's how to use np.r_ to construct a array with closer spacing at the ends, and wider in the middle:
In [578]: x=np.r_[0:.09:10j, .1:.9:11j, .91:1:10j]
In [579]: x
Out[579]:
array([ 0. , 0.01, 0.02, 0.03, 0.04, 0.05, 0.06, 0.07, 0.08,
0.09, 0.1 , 0.18, 0.26, 0.34, 0.42, 0.5 , 0.58, 0.66,
0.74, 0.82, 0.9 , 0.91, 0.92, 0.93, 0.94, 0.95, 0.96,
0.97, 0.98, 0.99, 1. ])
then scale x with np.pi.
This is the kind of thing that np.r_ was created for. Not that it's doing anything special. It's doing the same as:
np.concatenate([np.linspace(0,.09,10),
np.linspace(.1,.9,11),
np.linspace(.91,1,10)])
For a smoother gradation in spacing, I'd try mapping a single linspace with a curve.
In [606]: x=np.arctan(np.linspace(-10,10,10))
In [607]: x -= x[0]
In [608]: x /= x[-1]
In [609]: x
Out[609]:
array([ 0. , 0.00958491, 0.02665448, 0.06518406, 0.21519086,
0.78480914, 0.93481594, 0.97334552, 0.99041509, 1. ])