I would like to know the fastest way to multiply each column of a numpy array by one another and return a new numpy array consisting of these just-built columns - all 2-element combinations from n-element set of columns.
This should be a good starting point
import numpy as np
I, J = np.triu_indices(n)
A[:,I] * A[:,J]
Related
I am trying to come up with a fast way to calculate l2 distance between the rows of two 2d numpy arrays. So first 2d numpy array is 7000 x 100 and second 2d numpy array is 4000 x 100.
I want to get a matrix of 4000 x 7000, where each (i, j) entry is a l2 norm between ith row of second 2d numpy array and jth row of first 2d numpy array.
I am assuming I probably have to use numpy.linalg.norm, but am not quite sure on how to vectorize the operation.
Thanks for the help! Let me know if the question is unclear in any ways.
There is a function in scipy designed specifically for finding all distances between two sets of vectors:
scipy.spatial.distance.cdist(a, b)
I have created a 2D 10x10 Array. using Numpy I want to iterate over the array as efficiently as possible.
However I would like to return the array values. essentially iterating over the 10x10 array 10 times and return a 1x10 array each time.
import datetime
import numpy as np
import random
start = datetime.datetime.now()
a = np.random.uniform(low=-1, high=1, size=(10,10))
print("Time :",datetime.datetime.now() - start)
for x in np.nditer(a):
print(x)
the result is as follows:
0.5738994777717537
0.24988408410910767
0.8391827831682657
0.0015975845830569213
0.54477459840569
0.14091622639476165
-0.36517132895234106
-0.06311125453484467
-0.6572544506539948
...
100 times
However I would expect the result to be:
[0.5738994777717537,
0.24988408410910767,
0.8391827831682657,
0.0015975845830569213,
0.54477459840569,
0.14091622639476165,
-0.36517132895234106,
-0.06311125453484467,
-0.6572544506539948],[...]
...
10 times
Any help would be appreciated!
To directly answer your question, this does exactly what you want:
import numpy as np
a = np.random.uniform(low=-1, high=1, size=(10,10))
print(','.join([str(list(x)) for x in a]))
This will print
[-0.2403881196886386, ... , 0.8518165986395723],[-0.2403881196886386, ... , 0.8518165986395723], ..., [-0.2403881196886386, ... , 0.8518165986395723]
The reason you're printing just the elements of the array is due to the way nditer works. nditer iterates over single elements, even at a multidimensional level, whereas you want to iterate over just the first dimension of the array. For that, for x in a: works as intended.
Edit
Here is a good link if you want to read up on how nditer works: https://docs.scipy.org/doc/numpy/reference/arrays.nditer.html#arrays-nditer
I'm using Python 2.7.6
I have Three 2-dimensional matrices with the same columns(n) and the same rows(m). How can I collapse these 3 matrices into one megastructure that is the same indices and columns labels (m x n matrix). Right now they are DataFrames in Pandas.
#Pseudocode
import pandas as pd
import numpy as np
DF_1 = pd.DataFrame(data)
DF_2 = pd.DataFrame(data)
DF_3 = pd.DataFrame(data)
#I was thinking maybe a dot product like:
np.dot(DF_metNorm.as_matrix(),DF_cnvNorm.as_matrix(),DF_gexNorm.as_matrix())
#but np.dot() can only take two matrices
Is there any other way I can collapse these into one matrice that could include all of the data into a single metric using either numpy scipy sklearn or pandas? All values would be a scalar 2D matrix.
Whats wrong with a Panel? You could say
panel = pd.Panel({'df1' : DF_1,
'df2' : DF_2,
'df3' : DF_3})
and then have it together in one mega-structure with all of Pandas' awesome indexing and selection capabilities.
This is a pretty easy question, I was wondering how to decipher this array:
model[Best[i][j]][6]
Is it recreating another array based off of the 'Best' array within the brackets? I'm not sure how to translate this to myself.
If we are talking about numpy arrays, this will return the value of array model positioned at Best[i][j] (this should be a number perhaps from another array) row and 6th column. Here is an example:
import numpy as np
model = np.array([[1,2],[3,4]])
Best = np.array([[0,0],[1,1]])
i = 0 # Best[i][j] is 0
j = 1
print model[Best[i][j]][1] # It prints model[0][1], which is 2
I have a loop that adds elements to a 1d array:
for i in range(0, 1000):
fvector[0, i] = function_value
after the loop finishes, I have a 1 x 1000 vector that I want to store in a multi-dimensional array fmatrix, which is 50 x 1000. I managed to do this using a loop and copying each element individually - but it is very slow. I've then tried to use slice to copy the whole vector in one go after the loop and then be ready to copy next vector at the next column. How do I make it go to the next column? I've tried:
s=slice([i], None)
fmatrix[s] = fvector
and various combinations for s, but I get error messages about setting an array element with a sequence, or invalid syntax.
I know this should be straight forward but I'm very new to python, numpy and arrays :-(
Try this. Allocate the matrix, here zero-initialized for effect:
>>> import numpy as np
>>> fmatrix = np.zeros((50, 1000))
Then index into it to obtain fvector:
>>> fvector = fmatrix[0]
Then assign to fvector's elements:
>>> for i in xrange(1000):
... fvector[i] = i
If you now inspect fmatrix[0], the first row of fmatrix, you'll find that it has been assigned to in the previous loop. That's because the NumPy row indexing creates fvector as a view on fmatrix's first row. This saves you a copy.
fvector has shape (1,1000). That's a 2D array, even if one axis has length 1.
You can slice it down to a 1D array with fvector[0,:]. This gives the first row.
fmatrix has shape (50,1000). You can slice it down to a 1D array with fmatrix[i,:]. This gives the ith row.
So to assign the values in the first row of fvector to the ith row of fmatrix:
fmatrix[i,:] = fvector[0,:]
Perhaps however there is no need for fvector to be a 2D array? Perhaps just make it a 1D array to begin with:
fvector = np.empty(1000)
for i in range(0, 1000):
fvector[i] = function_value
and then you could do the assignment with
fmatrix[i,:] = fvector