I have an array of arrays that represent matrices and I need to transpose each matrix, ideally without transposing in a loop. When I use array.T, it transposes everything, not just the axes in each array. Is it possible to just transpose each matrix?
INPUT: np.arange(27).reshape(3, 3, 3).T
OUTPUT:
[[[ 0 9 18]
[ 3 12 21]
[ 6 15 24]]
[[ 1 10 19]
[ 4 13 22]
[ 7 16 25]]
[[ 2 11 20]
[ 5 14 23]
[ 8 17 26]]]
What I want is for the arrays to look like this:
[[[ 0 3 6]
[ 1 4 7]
[ 2 5 8]]
[[ 9 12 15]
[ 10 13 16]
[ 11 14 17]]
[[ 18 21 24]
[ 19 22 25]
[ 20 23 26]]]
In [11]: A = np.arange(27).reshape(3, 3, 3)
In [12]: A
Out[12]:
array([[[ 0, 1, 2],
[ 3, 4, 5],
[ 6, 7, 8]],
[[ 9, 10, 11],
[12, 13, 14],
[15, 16, 17]],
[[18, 19, 20],
[21, 22, 23],
[24, 25, 26]]])
transpose the last 2 dimensions:
In [13]: A.transpose(0,2,1)
Out[13]:
array([[[ 0, 3, 6],
[ 1, 4, 7],
[ 2, 5, 8]],
[[ 9, 12, 15],
[10, 13, 16],
[11, 14, 17]],
[[18, 21, 24],
[19, 22, 25],
[20, 23, 26]]])
A.swapaxes(1,2) also does it.
Related
I have a Spark dataframe:
> numbers_df
+----+-----------+-----------+-----------+-------------------------------------+
| id | num_1| num_2| num_3| all_num|
+----+-----------+-----------+-----------+-------------------------------------+
| 1| [1, 2, 5]| [4, 7]| [8, 3]| [1, 2, 3, 4, 5, 6, 7, 8, 9]|
| 2| [12, 13]| [10, 20]| [15, 17]| [10, 11, 12, 13, 14, 15, 16, 17, 18]|
+----+-----------+-----------+-----------+-------------------------------------+
I need to except from column all_num values of num_1, num_2 and num_3 columns.
Expected result:
id
num_1
num_2
num_3
all_num
except_num
1
[1, 2, 5]
[4, 7]
[8, 3]
[1, 2, 3, 4, 5, 6, 7, 8, 9]
[6, 9]
2
[12, 13]
[10, 16]
[15, 17]
[10, 11, 12, 13, 14, 15, 16, 17, 18]
[11, 14, 18]
How can this be done in PySpark? Since array_except function takes only two columns as input
You can combine array_except and concat functions.
df = df.withColumn('except_num', F.array_except('all_num', F.concat('num_1', 'num_2', 'num_3')))
df.show(truncate=False)
I have an array of size 300x5 and I am trying to sort the array in such a way that column with index 4 is my primary index and in ascending order, index 1 is secondary index and in descending order, index 3 is tertiary index and in ascending order.
I have tried this using following code,
idx = np.lexsort((arr[:,3],arr[:,1][::-1],arr[:,4]))
arr= arr[idx]
where arr --> array of size 300x5
On executing this the secondary index also gets sorted in ascending order instead of descending order. Can anyone help me with this
I think you want -arr[:,1] and not arr[:,1][::-1] as the secondary index.
>>> import numpy as np
>>> arr = np.random.randint(0, 21, (300, 5))
>>> arr
array([[ 0, 19, 6, 19, 17],
[16, 2, 14, 17, 0],
[ 8, 17, 3, 17, 12],
...,
[ 4, 18, 18, 3, 8],
[10, 15, 4, 12, 4],
[ 9, 16, 12, 0, 12]])
>>> idx = np.lexsort((arr[:,3],-arr[:,1],arr[:,4]))
>>> arr = arr[idx]
>>> arr
array([[11, 20, 11, 18, 0],
[11, 16, 12, 2, 0],
[ 9, 16, 4, 8, 0],
...,
[20, 4, 5, 11, 20],
[ 9, 4, 0, 19, 20],
[ 9, 2, 4, 10, 20]])
this is my first post here so please forgive me for any mistake.
This is an example:
My matrix:
[[0, 0, 0, 0],
[1, 1, 1, 1],
[2, 3, 4, 5],
[6, 7, 8, 9]]
My narray:
[20, 0, 10, 5]
Would like to get:
[[20, 0, 10, 5],
[21, 1, 11, 6],
[22, 3, 14, 10],
[26, 7, 18, 14]]
Which is the best way to do it?
Thatk you very much
You can just use simple + operator, numpy will do auto-broadcasting for you.
import numpy as np
a = np.array([[0,0,0,0],[1,1,1,1],[2,3,4,5],[6,7,8,9]])
b = np.array([20,0,10,5])
print(a+b)
# [[20 0 10 5]
# [21 1 11 6]
# [22 3 14 10]
# [26 7 18 14]]
USING IDLE/Python 3.5.1
May I first of all begin by saying I am a reasonably experienced programmer in VBA but am on day 2 of Python. I assure you I have conducted many searches on this question but the 30 or so documents I have read do not seem to explain my problem.
May I also please request that any answers given are properly formatted code for Python 3.5.1 rather than helpful pointers to other documentation or links?
The Problem
I am running a report and outputting results as I go. I need to store the results (presumably in an array) during this so that I can refer to them afterwards. The report (and the populating of the array) can be rerun multiple times so please bear that in mind if using concepts like 'append' when building the array. The array has dimensions [25,4] - a maximum of 25 records with four items in each.
Day X Y Z Total
1 2 3 4 9
2 3 4 5 12 ...
(Purists: The total needs to be recorded rather than calculated because of rounding.)
I could solve the problem myself if someone could translate this bit of code into Python (from VBA for illustration purposes). I do not want to import the arrays module unless it's the only way. Note: Variable l is a loop that makes the array get built twice to demonstrate that the array needs to be capable of rebuilding from scratch rather than being created just the once.
Sub sArray()
Dim a(25, 4)
For l = 1 To 2
For i = 1 To 25
For j = 1 To 4
a(i, j) = Int(100 * Rnd(1)) + 1
Debug.Print a(i, j);
Next j
Next i
Next l
End Sub
Thanks,
Tom
I am not sure I got your question correctly...
If you want to make an array (list i a better term in this case) of size [25,4] this is one way to go:
import random
a = [[int(100*random.random())+1 for j in range(4)] for i in range(25)]
>>> print a
[[74, 17, 36, 75],
[1, 79, 33, 90],
[58, 66, 47, 95],
[35, 40, 87, 38],
[43, 46, 34, 66],
[69, 34, 26, 49],
[56, 83, 44, 14],
[2, 44, 54, 97],
[50, 21, 39, 60],
[13, 94, 12, 48],
[36, 13, 2, 71],
[77, 44, 31, 11],
[56, 26, 30, 39],
[17, 13, 83, 84],
[54, 37, 34, 18],
[5, 54, 88, 100],
[22, 77, 70, 21],
[51, 88, 26, 97],
[69, 33, 86, 48],
[42, 66, 38, 78],
[71, 43, 96, 23],
[6, 46, 100, 29],
[32, 86, 15, 48],
[96, 84, 8, 56],
[29, 64, 69, 79]]
if you want to show that "the array needs to be capable of rebuilding from scratch rather than being created just the once" (why would you need this??)
for l in range(2):
a = [[int(100*random.random())+1 for j in range(4)] for i in range(25)]
Also, the way of generating random numbers is odd (I have translated you method). To get the same result in python, just use random.randint(1,100) to generate random integers from 1 (i think you do not want to have 0 there) to whatever number you like.
If I have correctly understood from your comments, this is what you want:
def report(g=25):
array = []
for _ in range(g):
x = random.randint(1,10)
y = random.randint(1,10)
z = random.randint(1,10)
total = x+y+x
row = [x,y,z,total]
print(row)
array.append(row)
return array
result = report()
#prints all the rows while computing
>>> result #stores the "array"
[8, 4, 3, 20]
[10, 7, 4, 27]
[2, 4, 5, 8]
[8, 5, 8, 21]
[9, 7, 2, 25]
[2, 2, 3, 6]
[5, 8, 6, 18]
[8, 6, 1, 22]
[7, 6, 4, 20]
[7, 2, 10, 16]
[6, 5, 9, 17]
[3, 8, 8, 14]
[9, 1, 9, 19]
[1, 7, 7, 9]
[6, 6, 2, 18]
[9, 10, 1, 28]
[4, 6, 2, 14]
[6, 1, 6, 13]
[4, 1, 3, 9]
[5, 3, 5, 13]
[7, 5, 2, 19]
[9, 5, 7, 23]
[2, 5, 8, 9]
[3, 10, 4, 16]
[5, 6, 5, 16]
If I have the following data:
A = np.random.random((3, 4, 5))
# np.all(indices < A.shape) is true
indices = np.array([
[0, 0, 0],
[1, 2, 4],
...
[2, 3, 4]
])
How can I use each row of indices as a set of axis indices into A to give the following?
B = np.array([
A[0, 0, 0],
A[1, 2, 4],
...
A[2, 3, 4]
])
Here's a 2d example:
In [1]: A=np.arange(10,22).reshape(3,4)
In [2]: A
Out[2]:
array([[10, 11, 12, 13],
[14, 15, 16, 17],
[18, 19, 20, 21]])
In [3]: ind=np.array([[0,1],[1,3],[2,0],[0,2]])
In [4]: ind
Out[4]:
array([[0, 1],
[1, 3],
[2, 0],
[0, 2]])
In [5]: A[ind[:,0],ind[:,1]]
Out[5]: array([11, 17, 18, 12])
or for your variables,
A[indices[:,0], indices[:,1], indices[:,2]]
Or more generally:
In [8]: tuple(ind.T)
Out[8]: (array([0, 1, 2, 0]), array([1, 3, 0, 2]))
In [9]: A[tuple(ind.T)]
Out[9]: array([11, 17, 18, 12])
This is based on the idea that A[a,b] is the same as A[(a,b)]. And when a and b are matching lists or arrays, it selects values by pairing them up, roughly the same as
[A[i,j] for i,j in zip(a,b)]
For a product like indexing, the index arrays need to have more dimensions. ix_ is a handy way of generating such arrays:
In [53]: np.ix_(ind[:,0],ind[:,1])
Out[53]:
(array([[0],
[1],
[2],
[0]]), array([[1, 3, 0, 2]]))
In [54]: A[np.ix_(ind[:,0],ind[:,1])]
Out[54]:
array([[11, 13, 10, 12],
[15, 17, 14, 16],
[19, 21, 18, 20],
[11, 13, 10, 12]])
In [56]: A[ind[:,[0]],ind[:,1]]
Out[56]:
array([[11, 13, 10, 12],
[15, 17, 14, 16],
[19, 21, 18, 20],
[11, 13, 10, 12]])
You could use np.ravel_multi_index to generate the linear indices and then extract the selective elements from A with linear-indexing using np.take like so -
np.take(A,np.ravel_multi_index(indices.T,A.shape))