If
X = np.array([[1, 1],[1, 2],[2, 2],[2, 3]])
and you do np.dot(X, np.array([1,2])), how does that multiply to become array([3, 5, 6, 8])?
I know X has a shape of (4,2) and the second array is (2,) (because it's 2 columns and it's 1D). I also know there is a special case from https://numpy.org/doc/stable/reference/generated/numpy.dot.html:
"If a is an N-D array and b is a 1-D array, it is a sum product over the last axis of a and b."
But I can't seem to apply it correctly to get the result.
In [376]: X = np.array([[1, 1],[1, 2],[2, 2],[2, 3]])
In [378]: y=np.array([1,2])
In [379]: X.shape
Out[379]: (4, 2)
In [380]: y.shape
Out[380]: (2,)
the dot/matmul approach:
In [381]: X#y
Out[381]: array([3, 5, 6, 8])
einsum lets us specify which axes combine how:
In [382]: np.einsum('ij,j->i',X,y)
Out[382]: array([3, 5, 6, 8])
elementwise multipication followed by sum. The (4,2) * (2,) -> (4,2)
In [383]: X*y
Out[383]:
array([[1, 2],
[1, 4],
[2, 4],
[2, 6]])
In [384]: (X*y).sum(1)
Out[384]: array([3, 5, 6, 8])
Or by highschool math, each row of X times y, summed
In [386]: [sum(row*y) for row in X]
Out[386]: [3, 5, 6, 8]
Related
How can I slice the last 2 columns of a numpy array?
for example:
A = np.array([[1, 2, 3], [4, 5, 6]])
and I want to get B as the last 2 columns of A which is [[2, 3], [5, 6]]
I know that I can index it from start of the array such as B = A[:, 1:3]. But I am looking for a general form to slice A by indexing from the end as the number of columns for A changes in my case.
Here you go
>>> A = np.array([[1, 2, 3],[4, 5, 6]])
>>> A[:,[-2,-1]]
array([[2, 3],
[5, 6]])
A generalised method to get the last n rows can be
>>> A = np.array([[1, 2, 3,4],[4, 5, 6,7]])
>>> A[:,-3:]
array([[2, 3, 4],
[5, 6, 7]])
I am in the following situation - I have the following:
Multidimensional numpy array a of n dimensions
t, an array of k rows (tuples), each with n elements. In other words, each row in this array is an index in a
What I want: from a, return an array b with k scalar elements, the ith element in b being the result of indexing a with the ith tuple from t.
Seems trivial enough. The following approach, however, does not work
def get(a, t):
# wrong result + takes way too long
return a[t]
I have to resort to doing this iteratively i.e. the following works correctly:
def get(a, t):
res = []
for ind in t:
a_scalar = a
for i in ind:
a_scalar = a_scalar[i]
# a_scalar is now a scalar
res.append(a_scalar)
return res
This works, except for the fact that given that each dimension in a has over 30 elements, the procedure does get really slow when n gets to more than 5. I understand that it would be slow regardless, however, I would like to exploit numpy's capabilities as I believe it would speed up this process considerably.
The key to getting this right is to understand the roles of indexing lists and tuples. Often the two are treated the same, but in numpy indexing, tuples, list and arrays convey different information.
In [1]: a = np.arange(12).reshape(3,4)
In [2]: t = np.array([(0,0),(1,1),(2,2)])
In [4]: a
Out[4]:
array([[ 0, 1, 2, 3],
[ 4, 5, 6, 7],
[ 8, 9, 10, 11]])
In [5]: t
Out[5]:
array([[0, 0],
[1, 1],
[2, 2]])
You tried:
In [6]: a[t]
Out[6]:
array([[[ 0, 1, 2, 3],
[ 0, 1, 2, 3]],
[[ 4, 5, 6, 7],
[ 4, 5, 6, 7]],
[[ 8, 9, 10, 11],
[ 8, 9, 10, 11]]])
So what's wrong with it? It ran, but selected a (3,2) array of rows of a. That is, it applied t to just the first dimension, effectively a[t, :]. You want to index on all dimensions, some sort of a[t1, t2]. That's the same as a[(t1,t2)] - a tuple of indices.
In [10]: a[tuple(t[0])] # a[(0,0)]
Out[10]: 0
In [11]: a[tuple(t[1])] # a[(1,1)]
Out[11]: 5
In [12]: a[tuple(t[2])]
Out[12]: 10
or doing all at once:
In [13]: a[(t[:,0], t[:,1])]
Out[13]: array([ 0, 5, 10])
Another way to write it, is n lists (or arrays), one for each dimension:
In [14]: a[[0,1,2],[0,1,2]]
Out[14]: array([ 0, 5, 10])
In [18]: tuple(t.T)
Out[18]: (array([0, 1, 2]), array([0, 1, 2]))
In [19]: a[tuple(t.T)]
Out[19]: array([ 0, 5, 10])
More generally, in a[idx1, idx2] array idx1 is broadcast against idx2 to produce a full selection array. Here the 2 arrays are 1d and match, the selection is your t set of pairs. But the same principle applies to selecting a set of rows and columns, a[ [[0],[2]], [0,2,3] ].
Using the ideas in [10] and following, your get could be sped up with:
In [20]: def get(a, t):
...: res = []
...: for ind in t:
...: res.append(a[tuple(ind)]) # index all dimensions at once
...: return res
...:
In [21]: get(a,t)
Out[21]: [0, 5, 10]
If t really was a list of tuples (as opposed to an array built from them), your get could be:
In [23]: tl = [(0,0),(1,1),(2,2)]
In [24]: [a[ind] for ind in tl]
Out[24]: [0, 5, 10]
Explore using np.ravel_multi_index
Create some test data
arr = np.arange(10**4)
arr.shape=10,10,10,10
t = []
for j in range(5):
t.append( tuple(np.random.randint(10, size = 4)))
print(t)
# [(1, 8, 2, 0),
# (2, 3, 3, 6),
# (1, 4, 8, 5),
# (2, 2, 6, 3),
# (0, 5, 0, 2),]
ta = np.array(t).T
print(ta)
# array([[1, 2, 1, 2, 0],
# [8, 3, 4, 2, 5],
# [2, 3, 8, 6, 0],
# [0, 6, 5, 3, 2]])
arr.ravel()[np.ravel_multi_index(tuple(ta), (10,10,10,10))]
# array([1820, 2336, 1485, 2263, 502]
np.ravel_multi_index basically calculates, from the tuple of input arrays, the index into a flattened array that starts with shape (in this case) (10, 10, 10, 10).
Does this do what you need? Is it fast enough?
I have an array of shape ( 2084, 2084) i want to reshape it to (2084, 2084 , 3). I tried using np.dstack but it gives me something like this (1, 2084, 2084)
patch = (2084, 2084)
patch_new = np.dstack(patch)
How do I do it?
You missed promoting your array to 3D before depth stacking. So, you can use something like:
In [93]: patch = (2084, 2084)
In [94]: arr = np.random.random_sample(patch)
# make it as 3D array
In [95]: arr = arr[..., np.newaxis]
# and then stack it along the third dimension (say `n` times; here `3`)
In [96]: arr_3d = np.dstack([arr]*3)
In [97]: arr_3d.shape
Out[97]: (2084, 2084, 3)
Another way to do the same is (i.e. if you don't wish to promote your input array explicitly to 3D):
In [140]: arr_3d = np.dstack([arr]*3)
In [141]: arr_3d.shape
Out[141]: (2084, 2084, 3)
# sanity check
In [146]: arr_3 = np.dstack([arr[..., np.newaxis]]*3)
In [147]: arr_3.shape
Out[147]: (2084, 2084, 3)
In [148]: np.allclose(arr_3, arr_3d)
Out[148]: True
In [730]: x = np.arange(8).reshape(2,4)
In [731]: x
Out[731]:
array([[0, 1, 2, 3],
[4, 5, 6, 7]])
Your dstack not only adds an initial dimension, it transposes the rest. That's because it treats your array as a list, np.dstack([x[0,:], x[1,:]]).
In [732]: np.dstack(x)
Out[732]:
array([[[0, 4],
[1, 5],
[2, 6],
[3, 7]]])
This is a repeat task
In [733]: np.repeat(x[...,None],3,axis=2)
Out[733]:
array([[[0, 0, 0],
[1, 1, 1],
[2, 2, 2],
[3, 3, 3]],
[[4, 4, 4],
[5, 5, 5],
[6, 6, 6],
[7, 7, 7]]])
Kmario, so you repeat the same array 3 times over the third dimension ?
I'm using Ruby 2.4. If I have an ordered array of numbers, say
[1, 2, 4, 7, 8, 9]
How do I find the numerical elements that aren't present in my array, between the smallest and greatest values in the array? For instance in the above, the missing values are
[3, 5, 6]
If my array were
[2, 7]
the missing values I'd be looking for would be
[3, 4, 5, 6]
Remove the existing numbers from the expected range of numbers:
(numbers.first..numbers.last).to_a - numbers
If you prefer complicated solutions:
[1, 2, 4, 7, 8, 9].chunk_while { |a, b| a + 1 == b }
.each_cons(2)
.flat_map { |x, y| (x.last + 1).upto(y.first - 1).to_a }
#=> [3, 5, 6]
I've been scanning the forums and haven't found an answer yet that I can apply to my situation. I need to be able to take an n by n array and transpose it in Python-3. The example given is that I have this list input into the function:
[[4, 2, 1], ["a", "a", "a"], [-1, -2, -3]] and it needs to be transposed to read:
[[4, 'a', -1], [2, 'a', -2], [1, 'a', -3]] So basically reading vertically instead of horizontally.
I CANNOT use things like zip or numpy, I have to make my own function.
Been rattling my brain at this for two nights and it's a huge headache. If anyone could help and then provide an explanation so I can learn it, I'd be grateful.
Edit:
I should add for reference sake that the argument variable is M. The function we're supposed to write is trans(M):
A one-liner:
def trans(M):
return [[M[j][i] for j in range(len(M))] for i in range(len(M[0]))]
result:
>>> M = [[1, 2, 3], [4, 5, 6], [7, 8, 9]]
>>> trans(M)
[[1, 4, 7], [2, 5, 8], [3, 6, 9]
# or for a non-square matrix:
>>> N = [[1, 2, 3], [4, 5, 6], [7, 8, 9], [10, 11, 12]]
>>> trans(N)
[[1, 4, 7, 10], [2, 5, 8, 11], [3, 6, 9, 12]]
Additional Note: If you look up the tutorial on list comprehension, one of the examples is in fact transposition of a matrix array.
A variant that should work for matrices with irregular row lengths:
m=[[3, 2, 1],
[0, 1],
[2, 1, 0]]
m_T = [ [row[c] for row in m if c < len(row)] for c in range(0, max([len(row) for row in m])) ]
Here is an in place solution that works for square matrices:
def trans(M):
n = len(M)
for i in range(n - 1):
for j in range(i + 1, n):
M[i][j], M[j][i] = M[j][i], M[i][j]
Example Usage:
def print_matrix(M):
for row in M:
for ele in row:
print(ele, end='\t')
print()
M = [[4, 2, 1], ["a", "a", "a"], [-1, -2, -3]]
print('Original Matrix:')
print_matrix(M)
trans(M)
print('Transposed Matrix:')
print_matrix(M)
Output:
Original Matrix:
4 2 1
a a a
-1 -2 -3
Transposed Matrix:
4 a -1
2 a -2
1 a -3
y=([1,2], [3,4], [5,6])
transpose=[[row[i] for row in y] for i in range(len(y[0]))]
the output is
[[1, 3, 5], [2, 4, 6]]
You can also use the function in numpy to transpose - if you need the answer as a list it is straightforward to convert back using tolist:
from numpy import transpose
M = [[1, 2, 3], [4, 5, 6], [7, 8, 9]]
transpose(M).tolist()
the output is
[[1, 4, 7], [2, 5, 8], [3, 6, 9]]
Haven't timed it (no time!) but I strongly suspect this will be a lot faster than iterators for large arrays, especially if you don't need to convert back to a list.