I am very amateur when it comes to scipy. I am trying to use scipy's fmin function on a multidimensional variable system. For the sake of simplicity I am using list of list of list's. My data is 12 dimensional, when I enter np.shape(DATA) it returns (3,2,2), I am not even sure if scipy can handle that many dimensions, if not no problem I can reduce them, the point is that the optimize.fmin() function doesn't accept list based arrays as x0 initial parameters, so I need help either rewriting the x0 array into numpy compatible one or the entire DATA array into a 12 dimensional matrix or something like that.
Here is a simpler example illustrating the issue:
from scipy import optimize
import numpy as np
def f(x): return(x[0][0]*1.5-x[0][1]*2.0+x[1][0]*2.5-x[1][1]*3.0)
result = optimize.fmin(f,[[0.1,0.1],[0.1,0.1]])
print(result)
It will give an error saying invalid index to scalar variable which probably comes from not understanding the [[],[]] list of list structure, so it probably only understands numpy array formats.
So how to rewrite this to make it work, and also for my (3,2,2) shaped list of list as well!?
scipy.optimize.fmin needs the initial guess for the function parameters to be a 1D array with a number of elements that suits the function to optimize. In your case, maybe you can use flatten and reshape if you just need the output to be in the same shape as your input parameters. An example based on your illustration code:
from scipy import optimize
import numpy as np
def f(x):
return x[0]*1.5-x[1]*2.0+x[2]*2.5-x[3]*3.0
guess = np.array([[0.1, 0.1],
[0.1, 0.1]]) # guess.shape is (2,2)
out = optimize.fmin(f, guess.flatten()) # flatten upon input
# out.shape is (4,)
# reshape output according to guess
out = out.reshape(guess.shape) # out.shape is (2,2) again
or out = optimize.fmin(f, guess.flatten()).reshape(guess.shape) in one line. Note that this also works for a 3-dimensional array as you propose:
guess = np.arange(12).reshape(3,2,2)
# array([[[ 0, 1],
# [ 2, 3]],
# [[ 4, 5],
# [ 6, 7]],
# [[ 8, 9],
# [10, 11]]])
guess = guess.flatten()
# array([ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11])
guess = guess.reshape(3,2,2)
# array([[[ 0, 1],
# [ 2, 3]],
# [[ 4, 5],
# [ 6, 7]],
# [[ 8, 9],
# [10, 11]]])
Related
I have an array in three dimensions (a, b, c) and I need to modify the positions c indexed by an array in two dimensions (a, b).
I wrote a code that works as expected, but does anyone know if there is a way to solve this problem without the for loop?
import numpy as np
arr = np.random.randn(100, 5, 2)
mod = np.random.randn(100, 5)
ids = np.random.randint(0, 2, size=[100,5])
for i in range(100):
for j in range(5):
arr[i,j,ids[i,j]] = mod[i,j]
You can refer to and set each slice of the array directly. I think this code shows the behaviour you are asking about:
import numpy as np
arr = np.array([[[1, 2, 3], [4, 5, 6]], [[7, 8, 9], [10, 11, 12]]])
print(arr[:,:,2])
# Output:
#[[ 3 6]
# [ 9 12]]
new2DSlice = np.array([[23, 26], [29, 32]])
arr[:,:,2] = new2DSlice
print(arr[:,:,2])
# Outupt:
#[[23 26]
# [29 32]]
arr[:,:,2] refers to the third slice of the array and, in this example, sets it directly.
You can read about NumPy's array indexing and array slicing on W3Schools.
I got with this:
import numpy as np
arr = np.random.randn(100, 5, 2)
mod = np.random.randn(100, 5)
ids = np.random.randint(0, 2, size=[100,5])
x = np.arange(100)
y = np.arange(5)
arr[x[:,None],y,ids] = mod
I am trying to squeeze as much performance as I can using numpy arrays.
I have a Look Up Table say for e.g. MYLUT = [0,2,5,7,8,12,-1,...,229]
And I have an numpy array MYARRAY with uint8 values of shape (300,200000)
What is the fastest way to apply the lookup table to all the elements in MYARRAY?
Currently I am just using for loop:
for i in range(300) :
for j in range(200000) :
MYARRAY[i][j] = MYLUT[MYARRAY[i][j]]
Can I do better? Or this is already the fastest way to do this in Python?
NumPy can do this very efficiently:
arr = np.array([[1,2,3],[0,0,1]])
lut = np.array([0,2,5,7,8,12])
print(lut.take(arr))
It gives:
array([[2, 5, 7],
[0, 0, 2]])
Note this does the same thing as lut[arr]. Also note you should pay attention to your dtypes--in your example data you have -1 in the LUT but you say your data is unsigned.
If you want to be extra-efficient, you can use the out parameter of take():
lut.take(arr, out=arr)
That avoids creating a new array, and is more similar to your original code which updates arr in place.
MYLUT[MYARRAY] should work:
MYLUT = np.array([0,2,5,7,8,12,-1,229])
MYARRAY = np.random.randint(0,8,8).reshape(4,2)
MYARRAY
#array([[5, 4],
# [7, 5],
# [4, 3],
# [1, 5]])
new_array = np.empty_like(MYARRAY)
for i in range(4) :
for j in range(2) :
new_array[i,j] = MYLUT[MYARRAY[i,j]]
new_array
#array([[ 12, 8],
# [229, 12],
# [ 8, 7],
# [ 2, 12]])
MYLUT[MYARRAY]
#array([[ 12, 8],
# [229, 12],
# [ 8, 7],
# [ 2, 12]])
I'm trying to write a function that will take as input a numpy array in the form:
a = [[0,0], [10,0], [10,10], [5,4]]
and return a numpy array b such that:
b = [[[0,0]], [[10,0]], [[10,10]], [[5,4]]]
For some reason I'm finding this surprisingly difficult.
The reason I'm doing this is that I have some contours generated using skimage that I'm attempting to use opencv2 on to calculate features ( area, perimeter etc...) but the opencv functions will only take arrays in the form of b as input, rather than a.
a is shape (4,2), b is (4,1,2)
a.reshape(4,1,2)
np.expanddims(a, 1)
a[:,None]
all work
In [503]: B
Out[503]:
array([[[ 0, 0]],
[[10, 0]],
[[10, 10]],
[[ 5, 4]]])
In [504]: B.tolist()
Out[504]: [[[0, 0]], [[10, 0]], [[10, 10]], [[5, 4]]]
I wrote a function to see if a matrix is symmetric or not:
def issymmetric(mat):
if(mat.shape[0]!=mat.shape[1]):
return 0
for i in range(mat.shape[0]):
for j in range(i):
if (mat[i][j]!=mat[j][i]):
return 0
return 1
It works well with built-in ndarrays e.g. numpy.ones:
import numpy as np
a=np.ones((5,5), int)
print issymmetric(a)
And with numpy arrays:
import numpy as np
a=np.array([[1, 2, 3], [2, 1 , 2], [3, 2, 1]])
print issymmetric(a)
But when it comes to numpy matrixes:
import numpy as np
a=np.matrix([[1, 2, 3], [2, 1 , 2], [3, 2, 1]])
print issymmetric(a)
It gaves me this error:
File "issymetry.py", line 9, in issymmetric
if (mat[i][j]!=mat[j][i]):
File "/usr/lib/python2.7/dist-packages/numpy/matrixlib/defmatrix.py", line 316, in __getitem__
out = N.ndarray.__getitem__(self, index)
IndexError: index 1 is out of bounds for axis 0 with size 1
shell returned 1
That's because There is no a[0][1]
a[0] is matrix([[1, 2, 3]]). a[0][0] is matrix([[1, 2, 3]]) too., but there is no a[0][1].
How can I fix this issue, without changing the matrix type, or the function?
In general, what is the proper way to read and update one specific cell of a numpy matrix?
It is best to use [i,j] style indexing in numpy. Often you can get by with [i][j] when using np.array, but not with np.matrix. Remember an np.matrix is always 2d.
In a shell construct a simple 2d array, and try different methods of indexing. Now try it with np.matrix arrays. Pay attention to the shape.
In [2]: A = np.arange(6).reshape(2,3)
In [3]: A[1] # short for A[1,:]
Out[3]: array([3, 4, 5]) # shape (3,)
In [4]: A[1][2] # short for A[1,:][2]
Out[4]: 5
In [5]: M=np.matrix(A)
In [6]: M[1]
Out[6]: matrix([[3, 4, 5]]) # shape (1,3), 2d
In [7]: M[1][2]
...
IndexError: index 2 is out of bounds for axis 0 with size 1
correct indexing that works with both
In [9]: A[1,2]
Out[9]: 5
In [10]: M[1,2]
Out[10]: 5
A[i][j]=... is also prone to failure when used on the LHS. It only works if the first part A[i] returns a view. If fails if it produces a copy.
I have an numpy 2D array and I want it to return coloumn c where (r, c-1) (row r, coloumn c) equals a certain value (int n).
I don't want to iterate over the rows writing something like
for r in len(rows):
if array[r, c-1] == 1:
store array[r,c]
, because there are 4000 of them and this 2D array is just one of 20 i have to look trough.
I found "filter" but don't know how to use it (Found no doc).
Is there an function, that provides such a search?
I hope I understood your question correctly. Let's say you have an array a
a = array(range(7)*3).reshape(7, 3)
print a
array([[0, 1, 2],
[3, 4, 5],
[6, 0, 1],
[2, 3, 4],
[5, 6, 0],
[1, 2, 3],
[4, 5, 6]])
and you want to extract all lines where the first entry is 2. This can be done like this:
print a[a[:,0] == 2]
array([[2, 3, 4]])
a[:,0] denotes the first column of the array, == 2 returns a Boolean array marking the entries that match, and then we use advanced indexing to extract the respective rows.
Of course, NumPy needs to iterate over all entries, but this will be much faster than doing it in Python.
Numpy arrays are not indexed. If you need to perform this specific operation more effeciently than linear in the array size, then you need to use something other than numpy.