Related
I would like to create a vector of the same matrix in numpy (so as an array). Let's say the matrix is:
w = np.array([[1,2],
[3,4],
[5,6]])
Then, how can I create a vector of a fixed length with the matrix w in every position?
That is: vector[0] = ... = vector[n] = w
Not sure of the exact expected output, to to create an additional dimension you could use numpy.tile:
n = 3
vector = np.tile(w, (n, 1, 1))
NB. vector is not really a (1D) vector it is a 3D array
output:
# vector
array([[[1, 2],
[3, 4],
[5, 6]],
[[1, 2],
[3, 4],
[5, 6]],
[[1, 2],
[3, 4],
[5, 6]]])
# vector[0]
array([[1, 2],
[3, 4],
[5, 6]])
I am trying to implement an array of 3D vectors. All vectors are combination of element ranges. What I mean is:
array = [v_1, v_2, v_3,....]
v_j = [x_1, x_2, x_3] with x_i in [a, b].
The important thing for me is, that I want to have all possible combinations.
So for example let a = 1, b = 10. Then it should be something like:
v_1 = [1, 1, 1], v_2 = [1, 1, 2],...v_10 = [1, 1, 10]
and then the next one should be:
v_11 = [1, 2, 1], v_12 = [1, 2, 2]....
I tried it by using linspace but I just get the vectors where each element is equal i.e.
v_1 = [1, 1, 1], v_2 = [2, 2, 2]....
Is there an easy way to do this or do I have to do it by a lot of loops.
My linspace example was:
ffac = np.linspace(-1E-3, 1E-3, 100, endpoint=True)
for i in range(100):
eps = np.ones(shape=[100, ]) * ffac[i]
With a and b, we can make np.arange(a, b+1), and then use np.meshgrid:
xij = np.arange(a, b+1)
np.transpose(np.meshgrid(xij, xij, xij), (2,1,3,0))
For b=2, we obtain:
>>> np.transpose(np.meshgrid(xij, xij, xij), (2,1,3,0))
array([[[[1, 1, 1],
[1, 1, 2]],
[[1, 2, 1],
[1, 2, 2]]],
[[[2, 1, 1],
[2, 1, 2]],
[[2, 2, 1],
[2, 2, 2]]]])
For a vector of n options, the result is thus a n×n×n×3.
Or if you want to flatten it:
>>> np.transpose(np.meshgrid(xij, xij, xij), (2,1,3,0)).reshape(-1, 3)
array([[1, 1, 1],
[1, 1, 2],
[1, 2, 1],
[1, 2, 2],
[2, 1, 1],
[2, 1, 2],
[2, 2, 1],
[2, 2, 2]])
I'd like to prevent producing pairs with the same items when producing a random set of pairs in a Ruby array.
For example:
[1,1,2,2,3,4].shuffle.each_slice(2).to_a
might produce:
[[1, 1], [3, 4], [2, 2]]
I'd like to be able to ensure that it produces a result such as:
[[4, 1], [1, 2], [3, 2]]
Thanks in advance for the help!
arr = [1,1,2,2,3,4]
loop do
sliced = arr.shuffle.each_slice(2).to_a
break sliced if sliced.none? { |a| a.reduce(:==) }
end
Here are three ways to produce the desired result (not including the approach of sampling repeatedly until a valid sample is found). The following array will be used for illustration.
arr = [1,4,1,2,3,2,1]
Use Array#combination and Array#sample
If pairs sampled were permitted to have the same number twice, the sample space would be
arr.combination(2).to_a
#=> [[1, 4], [1, 1], [1, 2], [1, 3], [1, 2], [1, 1], [4, 1], [4, 2],
# [4, 3], [4, 2], [4, 1], [1, 2], [1, 3], [1, 2], [1, 1], [2, 3],
# [2, 2], [2, 1], [3, 2], [3, 1], [2, 1]]
The pairs containing the same value twice--here [1, 1] and [2, 2]--are not wanted so they are simple removed from the above array.
sample_space = arr.combination(2).reject { |x,y| x==y }
#=> [[1, 4], [1, 2], [1, 3], [1, 2], [4, 1], [4, 2], [4, 3],
# [4, 2], [4, 1], [1, 2], [1, 3], [1, 2], [2, 3], [2, 1],
# [3, 2], [3, 1], [2, 1]]
We evidently are to sample arr.size/2 elements from sample_space. Depending on whether this is to be done with or without replacement we would write
sample_space.sample(arr.size/2)
#=> [[4, 3], [1, 2], [1, 3]]
for sampling without replacement and
Array.new(arr.size/2) { sample_space.sample }
#=> [[1, 3], [4, 1], [2, 1]]
for sampling with replacement.
Sample elements of each pair sequentially, Method 1
This method, like the next, can only be used to sample with replacement.
Let's first consider sampling a single pair. We could do that by selecting the first element of the pair randomly from arr, remove all instances of that element in arr and then sample the second element from what's left of arr.
def sample_one_pair(arr)
first = arr.sample
[first, second = (arr-[first]).sample]
end
To draw a sample of arr.size/2 pairs we there execute the following.
Array.new(arr.size/2) { sample_one_pair(arr) }
#=> [[1, 2], [4, 3], [1, 2]]
Sample elements of each pair sequentially, Method 2
This method is a very fast way of sampling large numbers of pairs with replacement. Like the previous method, it cannot be used to sample without replacement.
First, compute the cdf (cumulative distribution function) for drawing an element of arr at random.
counts = arr.group_by(&:itself).transform_values { |v| v.size }
#=> {1=>3, 4=>1, 2=>2, 3=>1}
def cdf(sz, counts)
frac = 1.0/sz
counts.each_with_object([]) { |(k,v),a|
a << [k, frac * v + (a.empty? ? 0 : a.last.last)] }
end
cdf_first = cdf(arr.size, counts)
#=> [[1, 0.429], [4, 0.571], [2, 0.857], [3, 1.0]]
This means that there is a probability of 0.429 (rounded) of randomly drawing a 1, 0.571 of drawing a 1 or a 4, 0.857 of drawing a 1, 4 or 2 and 1.0 of drawing one of the four numbers. We therefore can randomly sample a number from arr by obtaining a (pseudo-) random number between zero and one (p = rand) and then determine the first element of counts_cdf, [n, q] for which p <= q:
def draw_random(cdf)
p = rand
cdf.find { |n,q| p <= q }.first
end
draw_random(counts_cdf) #=> 1
draw_random(counts_cdf) #=> 4
draw_random(counts_cdf) #=> 1
draw_random(counts_cdf) #=> 1
draw_random(counts_cdf) #=> 2
draw_random(counts_cdf) #=> 3
In simulation models, incidentally, this is the standard way of generating pseudo-random variates from discrete probability distributions.
Before drawing the second random number of the pair we need to modify cdf_first to reflect that fact that the first number cannot be drawn again. Assuming there will be many pairs to generate randomly, it is most efficient to construct a hash cdf_second whose keys are the first values drawn randomly for the pair and whose values are the corresponding cdf's.
cdf_second = counts.keys.each_with_object({}) { |n, h|
h[n] = cdf(arr.size - counts[n], counts.reject { |k,_| k==n }) }
#=> {1=>[[4, 0.25], [2, 0.75], [3, 1.0]],
# 4=>[[1, 0.5], [2, 0.833], [3, 1.0]],
# 2=>[[1, 0.6], [4, 0.8], [3, 1.0]],
# 3=>[[1, 0.5], [4, 0.667], [2, 1.0]]}
If, for example, a 2 is drawn for the first element of the pair, the probability is 0.6 of drawing a 1 for the second element, 0.8 of drawing a 1 or 4 and 1.0 of drawing a 1, 4, or 3.
We can then sample one pair as follows.
def sample_one_pair(cdf_first, cdf_second)
first = draw_random(cdf_first)
[first, draw_random(cdf_second[first])]
end
As before, to sample arr.size/2 values with replacement, we execute
Array.new(arr.size/2) { sample_one_pair }
#=> [[2, 1], [3, 2], [1, 2]]
With replacement, you may get results like:
unique_pairs([1, 1, 2, 2, 3, 4]) # => [[4, 1], [1, 2], [1, 3]]
Note that 1 gets chosen three times, even though it's only in the original array twice. This is because the 1 is "replaced" each time it's chosen. In other words, it's put back into the collection to potentially be chosen again.
Here's a version of Cary's excellent sample_one_pair solution without replacement:
def unique_pairs(arr)
dup = arr.dup
Array.new(dup.size / 2) do
dup.shuffle!
first = dup.pop
second_index = dup.rindex { |e| e != first }
raise StopIteration unless second_index
second = dup.delete_at(second_index)
[first, second]
end
rescue StopIteration
retry
end
unique_pairs([1, 1, 2, 2, 3, 4]) # => [[4, 3], [1, 2], [2, 1]]
This works by creating a copy of the original array and deleting elements out of it as they're chosen (so they can't be chosen again). The rescue/retry is in there in case it becomes impossible to produce the correct number of pairs. For example, if [1, 3] is chosen first, and [1, 4] is chosen second, it becomes impossible to make three unique pairs because [2, 2] is all that's left; the sample space is exhausted.
This should be slower than Cary's solution (with replacement) but faster (on average) than the posted solutions (without replacement) that require looping and retrying. Welp, chalk up another point for "always benchmark!" I was wrong about all most of my assumptions. Here are the results on my machine with an array of 16 numbers ([1, 1, 2, 2, 3, 4, 5, 5, 5, 6, 7, 7, 8, 9, 9, 10]):
cary_with_replacement
93.737k (± 2.9%) i/s - 470.690k in 5.025734s
mwp_without_replacement
187.739k (± 3.3%) i/s - 943.415k in 5.030774s
mudasobwa_without_replacement
129.490k (± 9.4%) i/s - 653.150k in 5.096761s
EDIT: I've updated the above solution to address Stefan's numerous concerns. In hindsight, the errors are obvious and embarrassing! On the plus side, the revised solution is now faster than mudasobwa's solution, and I've confirmed that the two solutions have the same biases.
You can check if there any mathes and shuffle again:
a = [1,1,2,2,3,4]
# first time shuffle
sliced = a.shuffle.each_slice(2).to_a
# checking if there are matches and shuffle if there are
while sliced.combination(2).any? { |a, b| a.sort == b.sort } do
sliced = a.shuffle.each_slice(2).to_a
end
It is unlikely, be aware about possibility of infinity loop
i dont understand why when i print the list of matrices, it prints out these random values! any insight would be great! thanks
matList = np.empty([2,2,2], dtype = int)
a = np.array([[1, 0],
[3, 3]])
b = np.array([[1, 1],
[3, 3]])
np.append(matList,a)
np.append(matList,b)
print (a)
print (b)
print (matList)
this is the output im getting
[[1 0]
[3 3]]
[[1 1]
[3 3]]
[[[6029427 4259908]
[3866700 3801155]]
[[5242972 7274610]
[7471207 7143521]]]
empty returns an array of the specified size, but filled 'random' values, what ever happened to be in those memory slots:
In [936]: mat = np.empty([2,2,2], dtype=int)
In [937]: mat
Out[937]:
array([[[ 0, -1231162112],
[-1222623584, 139401936]],
[[ 139401936, 139401936],
[-1230408992, -1222184576]]])
In [938]: a = np.array([[1, 0],
...: [3, 3]])
...: b = np.array([[1, 1],
...: [3, 3]])
...:
np.append not only returns a new array, but without axis flattens both inputs - the result is a 1d array:
In [939]: np.append(mat,a)
Out[939]:
array([ 0, -1231162112, -1222623584, 139401936, 139401936,
139401936, -1230408992, -1222184576, 1, 0,
3, 3])
What exactly are you trying to produce?
With a list append (and initial 'empty' list) (np.empty array does not mean the same thing):
In [941]: alist = []
In [942]: alist.append(a)
In [943]: alist.append(b)
In [944]: alist
Out[944]:
[array([[1, 0],
[3, 3]]), array([[1, 1],
[3, 3]])]
In [945]: np.array(alist)
Out[945]:
array([[[1, 0],
[3, 3]],
[[1, 1],
[3, 3]]])
The result is a (2,2,2) array, same as np.array([a,b])
You could also produce this by setting elements of mat with a and b (I would normally say by inserting, but I don't want to confuse you with np.insert):
In [951]: mat = np.empty((2,2,2), int)
In [952]: mat[0,:,:]=a
In [953]: mat[1,:,:]=b
In [954]: mat
Out[954]:
array([[[1, 0],
[3, 3]],
[[1, 1],
[3, 3]]])
This changes the values of mat in-place.
I need to get all submatrices of the 2D array and to do the manipulation for each submatrix. So I created example matrix:
M3 = [list(range(5)) for i in range(6)]
[[0, 1, 2, 3, 4],
[0, 1, 2, 3, 4],
[0, 1, 2, 3, 4],
[0, 1, 2, 3, 4],
[0, 1, 2, 3, 4],
[0, 1, 2, 3, 4]]
I need to capture 3 rows and 3 columns and then shift this "window" till I get all submatrices. The first submatrix would be:
[[0, 1, 2],
[0, 1, 2],
[0, 1, 2]]
and the last one is:
[[2, 3, 4],
[2, 3, 4],
[2, 3, 4]]
For this matrix I need 12 submatrices. However, I become more using code with which I tried to solve the problem:
for j in range(len(M3[0])-3):
for i in range(len(M3)-3):
for row in M3[0+j:3+j]:
X_i_j = [row[0+i:3+i] for row in M3[0+j:3+j]]
print(X_i_j)
I get 18 but not 12 (with two duplicates of each submatrix):
[[0, 1, 2], [0, 1, 2], [0, 1, 2]]
[[0, 1, 2], [0, 1, 2], [0, 1, 2]]
[[0, 1, 2], [0, 1, 2], [0, 1, 2]]
[[1, 2, 3], [1, 2, 3], [1, 2, 3]]
[[1, 2, 3], [1, 2, 3], [1, 2, 3]]
[[1, 2, 3], [1, 2, 3], [1, 2, 3]]
...
[[2, 3, 4], [2, 3, 4], [2, 3, 4]]
[[2, 3, 4], [2, 3, 4], [2, 3, 4]]
And with this sample of code I get 6 submatrices with 1 duplicate for each:
for i in range(len(M3)-3):
for j in range(len(M3[0])-3):
X_i_j = [row[0+i:3+i] for row in M3[0+j:3+j]]
print(X_i_j)
I do not see what is wrong with it and why I get the duplicates. How can I get all sub matrices of 2D array without numpy for this case?
Your code is working ( with change of order of vars and constants ):
for j in range(len(M3)-2):
for i in range(len(M3[0])-2):
X_i_j = [row[0+i:3+i] for row in M3[0+j:3+j]]
print('=======')
for x in X_i_j:
print(x)
I would solve it slightly different.
a function to read y-number-of-rows
then a function to read x-number-of-columns from those rows, which then is your sub.
This would work for any (2D) array / sub-array
Sample:
def read_y_rows(array, rows, offset):
return array[offset:rows + offset]
def read_x_cols(array, cols, offset):
return list(row[offset:cols + offset] for row in array)
def get_sub_arrays(array, x_dim_cols, y_dim_rows):
"""
get 2D sub arrays by x_dim columns and y_dim rows
from 2D array (list of lists)
"""
result = []
for start_row in range(len(array) - y_dim_rows + 1):
y_rows = read_y_rows(array, y_dim_rows, start_row)
for start_col in range(len(max(array, key=len)) - x_dim_cols + 1):
x_columns = read_x_cols(y_rows, x_dim_cols, start_col)
result.append(x_columns)
return result
to use it you could do:
M3 = [list(range(5)) for i in range(6)]
sub_arrays = get_sub_arrays(M3, 3, 3) ## this would also work for 2x2 arrays
the sub_arrays is again a list of lists, containing all found subarrays, you could print them like this:
for sub_array in sub_arrays:
print()
for row in sub_array:
print(row)
I know it is a lot more code than above, just wanted to share this code.