Related
I am in the following situation - I have the following:
Multidimensional numpy array a of n dimensions
t, an array of k rows (tuples), each with n elements. In other words, each row in this array is an index in a
What I want: from a, return an array b with k scalar elements, the ith element in b being the result of indexing a with the ith tuple from t.
Seems trivial enough. The following approach, however, does not work
def get(a, t):
# wrong result + takes way too long
return a[t]
I have to resort to doing this iteratively i.e. the following works correctly:
def get(a, t):
res = []
for ind in t:
a_scalar = a
for i in ind:
a_scalar = a_scalar[i]
# a_scalar is now a scalar
res.append(a_scalar)
return res
This works, except for the fact that given that each dimension in a has over 30 elements, the procedure does get really slow when n gets to more than 5. I understand that it would be slow regardless, however, I would like to exploit numpy's capabilities as I believe it would speed up this process considerably.
The key to getting this right is to understand the roles of indexing lists and tuples. Often the two are treated the same, but in numpy indexing, tuples, list and arrays convey different information.
In [1]: a = np.arange(12).reshape(3,4)
In [2]: t = np.array([(0,0),(1,1),(2,2)])
In [4]: a
Out[4]:
array([[ 0, 1, 2, 3],
[ 4, 5, 6, 7],
[ 8, 9, 10, 11]])
In [5]: t
Out[5]:
array([[0, 0],
[1, 1],
[2, 2]])
You tried:
In [6]: a[t]
Out[6]:
array([[[ 0, 1, 2, 3],
[ 0, 1, 2, 3]],
[[ 4, 5, 6, 7],
[ 4, 5, 6, 7]],
[[ 8, 9, 10, 11],
[ 8, 9, 10, 11]]])
So what's wrong with it? It ran, but selected a (3,2) array of rows of a. That is, it applied t to just the first dimension, effectively a[t, :]. You want to index on all dimensions, some sort of a[t1, t2]. That's the same as a[(t1,t2)] - a tuple of indices.
In [10]: a[tuple(t[0])] # a[(0,0)]
Out[10]: 0
In [11]: a[tuple(t[1])] # a[(1,1)]
Out[11]: 5
In [12]: a[tuple(t[2])]
Out[12]: 10
or doing all at once:
In [13]: a[(t[:,0], t[:,1])]
Out[13]: array([ 0, 5, 10])
Another way to write it, is n lists (or arrays), one for each dimension:
In [14]: a[[0,1,2],[0,1,2]]
Out[14]: array([ 0, 5, 10])
In [18]: tuple(t.T)
Out[18]: (array([0, 1, 2]), array([0, 1, 2]))
In [19]: a[tuple(t.T)]
Out[19]: array([ 0, 5, 10])
More generally, in a[idx1, idx2] array idx1 is broadcast against idx2 to produce a full selection array. Here the 2 arrays are 1d and match, the selection is your t set of pairs. But the same principle applies to selecting a set of rows and columns, a[ [[0],[2]], [0,2,3] ].
Using the ideas in [10] and following, your get could be sped up with:
In [20]: def get(a, t):
...: res = []
...: for ind in t:
...: res.append(a[tuple(ind)]) # index all dimensions at once
...: return res
...:
In [21]: get(a,t)
Out[21]: [0, 5, 10]
If t really was a list of tuples (as opposed to an array built from them), your get could be:
In [23]: tl = [(0,0),(1,1),(2,2)]
In [24]: [a[ind] for ind in tl]
Out[24]: [0, 5, 10]
Explore using np.ravel_multi_index
Create some test data
arr = np.arange(10**4)
arr.shape=10,10,10,10
t = []
for j in range(5):
t.append( tuple(np.random.randint(10, size = 4)))
print(t)
# [(1, 8, 2, 0),
# (2, 3, 3, 6),
# (1, 4, 8, 5),
# (2, 2, 6, 3),
# (0, 5, 0, 2),]
ta = np.array(t).T
print(ta)
# array([[1, 2, 1, 2, 0],
# [8, 3, 4, 2, 5],
# [2, 3, 8, 6, 0],
# [0, 6, 5, 3, 2]])
arr.ravel()[np.ravel_multi_index(tuple(ta), (10,10,10,10))]
# array([1820, 2336, 1485, 2263, 502]
np.ravel_multi_index basically calculates, from the tuple of input arrays, the index into a flattened array that starts with shape (in this case) (10, 10, 10, 10).
Does this do what you need? Is it fast enough?
I am trying to find the common elements in two arrays.
pairs = Array.new
a = exchange_one.get_symbols
b = exchange_two.get_symbols
c = a+b
c.uniq{|pair| pairs << pair}
I am combining the two arrays using +
Then I am calling uniq to remove the duplicate, but passing it to a block so the found duplicates can be added to an array before they are deleted.
For some reason the array pairs is just the entire c array.
What is the correct way to find array similarities.
If your goal is simply to determine which elements are the same between two arrays, you can use the intersection operator Array#&.
a = exchange_one.get_symbols
b = exchange_two.get_symbols
intersection = a & b
First understand what are you doing and what you want.
For eg.
a = 15.times.map { rand 6 }
#=> [1, 0, 5, 3, 1, 3, 4, 1, 3, 2, 1, 2, 4, 2, 3]
b = 15.times.map { rand 6 }
#=> [3, 3, 3, 1, 3, 1, 3, 1, 5, 1, 4, 2, 0, 0, 4]
Now what are you doing
c = a + b
#=> [1, 0, 5, 3, 1, 3, 4, 1, 3, 2, 1, 2, 4, 2, 3, 3, 3, 3, 1, 3, 1, 3, 1, 5, 1, 4, 2, 0, 0, 4]
c - only combine arrays irrespective of content hence get all values.
Now
pairs = Array.new
c.uniq{|pair| pairs << pair}
Here uniq is just act as a iterator means if you check 'pair' then it iterate all the values of 'c' and insert those values in 'pairs' array.
check this
c.uniq{|pair| puts pair}
Thats why you are getting all values within 'pairs' array.
The best way to find similarity in arrays is (a&b), but you can make changes in your code as follow to achieve it.
pairs = (arr1+arr2).uniq
OR
pairs = arr1 & arr2 #best and efficient way.
Suppose:
arr1 = 15.times.map { rand 6 }
#=> [1, 0, 4, 0, 2, 3, 1, 0, 2, 4, 4, 1, 3, 1, 1]
arr2 = 15.times.map { rand 6 }
#=> [5, 5, 4, 1, 5, 1, 5, 0, 4, 0, 2, 0, 4, 5, 0]
arr1 contains 5 1s and arr2 contains 2 1s. If, by "common elements" you wish to report that both arrays contain [5, 2].min #=> 2 1s, and similar counts for the other elements that appear in either array, you can do the following:
h1 = count(arr1)
#=> {1=>5, 0=>3, 4=>3, 2=>2, 3=>2}
h2 = count(arr2)
#=> {5=>5, 4=>3, 1=>2, 0=>4, 2=>1}
(h1.keys | h2.keys).each_with_object({}) { |k,h| h[k] = [h1[k], h2[k]].min }
#=> {1=>2, 0=>3, 4=>3, 2=>1, 3=>0, 5=>0}
def count(arr)
arr.each_with_object(Hash.new(0)) { |n,h| h[n] += 1 }
end
I want to create two sub-arrays from this array:
a = [0, 1, 2, 3, 4, 5, 6]
This array will not always contain the same number of elements because it depends on the user input.
For example, in some occasions it'll be:
a = [0, 5]
or:
a = [5, 6, 4]
I want to divide the array into two subarrays. The first one will contain numbers from 1 to 4 (inclusive) and the second one will contain 0, 5 and 6.
In the first example, it will be:
a = [0, 1, 2, 3, 4, 5, 6]
sub_array1 = [1, 2, 3, 4]
sub_array2 = [0, 5, 6]
In the second:
a = [0, 5]
sub_array1 = []
sub_array2 = [5]
In the third:
a = [5, 6, 4]
sub_array1 = [4]
sub_array2 = [5, 6]
and so on, depending on the user input.
How can I do this?
First thing that comes to mind is Enumerable#partition.
sub_array1, sub_array2 = [0,1,2,3,4,5,6].partition {|x| (1..4).include? x }
=> [[1,2,3,4], [0,5,6]]
if you have two conditions (I mean if 0,5,6 are an actual condition and not the excluded set) I think that a double iteration wouldn't hurt
a = [0,1,2,3,4,5,6]
sub_array1 = a.select { |x| (1..4).include? x }
sub_array2 = a.select { |x| [0,5,6].include? x }
You can try something like this:
[0,1,2,3,4,5,6].group_by{|x| [0,5,6].include? x}
The result will be a hash:
{true=>[0, 5, 6], false=>[1, 2, 3, 4]}
In the second case:
[0,5].group_by{|x| [0,5,6].include? x}
The result will be:
{true=>[0, 5]}
I'm switching from Matlab/octve to Numpy/Scipy.
To select a segment of a Matlab array, it was quite easy.
e.g.
>> x = [1, 2, 3, 4; 5, 6, 7, 8; 9, 10, 11, 12]
x =
1 2 3 4
5 6 7 8
9 10 11 12
>> y = x(2:3, 1:2)
y =
5 6
9 10
How can the same thing be done with NumPy when
x = np.array([[1, 2, 3, 4], [5, 6, 7, 8], [9, 10, 11, 12]])
As Indexing > Other indexing options in the NumPy documentation mentions,
The slicing and striding works exactly the same way it does for lists and tuples except that they can be applied to multiple dimensions as well.
For your example, this means
import numpy as np
x = np.array([[1, 2, 3, 4], [5, 6, 7, 8], [9, 10, 11, 12]])
# array([[ 1, 2, 3, 4],
# [ 5, 6, 7, 8],
# [ 9, 10, 11, 12]])
x[1:3, 0:2]
# => array([[ 5, 6],
# [ 9, 10]])
Most notable difference to Matlab is probably that indexing is zero-based (i.e., first element has index 0) and that index ranges (called 'slices' in Python) are expressed with an exclusive upper bound: l[4:7] gets l[4], l[5] and l[6] (the 3rd to the 7th element), but not l[7] (the 8th element).
The Python tutorial's section on lists will give you a feeling for how indexing and slicing works for normal (1-dimensional) collections.
This is my code, which works, but it's too big. I want to refactor it.
req_row = -1
req_col = -1
a.each_with_index do |row, index|
row.each_with_index do |col, i|
if col == 0
req_row = index
req_col = i
break
end
end
end
if req_col > -1 and req_row > -1
a.each_with_index do |row,index|
row.each_with_index do |col, i|
print (req_row == index or i == req_col) ? 0 : col
print " "
end
puts "\r"
end
end
Input: 2D Array
1 2 3 4
5 6 7 8
9 10 0 11
12 13 14 15
Required output:
1 2 0 4
5 6 0 8
0 0 0 0
12 13 0 15
I'm surprised the Matrix class is not used more:
a = [[ 1, 2, 3, 4],
[ 5, 6, 7, 8],
[ 9, 10, 0, 11],
[12, 13, 14, 15]]
require 'matrix'
m = Matrix.rows(a)
#=> Matrix[[1, 2, 3, 4], [5, 6, 7, 8], [9, 10, 0, 11], [12, 13, 14, 15]]
r, c = m.index(0)
#=> [2, 2]
Matrix.build(m.row_count, m.column_count) {|i,j| (i==r || j==c) ? 0 : m[i,j]}.to_a
#=> [[ 1, 2, 0, 4],
# [ 5, 6, 0, 8],
# [ 0, 0, 0, 0],
# [12, 13, 0, 15]]
Note Matrix objects are immutable. To change individual elements you must create a new matrix.
A slight modification is required if you wish to do this for every zero in the matrix:
a = [[ 1, 2, 3, 4],
[ 5, 6, 7, 8],
[ 9, 10, 0, 11],
[ 0, 13, 14, 15]]
require 'set'
m = Matrix.rows(a)
#=> Matrix[[1, 2, 3, 4], [5, 6, 7, 8], [9, 10, 0, 11], [0, 13, 14, 15]]
zero_rows = Set.new
zero_columns = Set.new
m.each_with_index { |e,i,j| (zero_rows << i; zero_columns << j) if e.zero? }
zero_rows
#=> #<Set: {2, 3}>
zero_columns
#=> #<Set: {2, 0}>
Matrix.build(m.row_count, m.column_count) do |i,j|
(zero_rows.include?(i) || zero_columns.include?(j)) ? 0 : m[i,j]
end.to_a
#=> [[0, 2, 0, 4],
# [0, 6, 0, 8],
# [0, 0, 0, 0],
# [0, 0, 0, 0]]
Try this code:
req_row = req_col = -1
a.each_with_index do |row, index|
req_col = row.index(0) # searching index having value 0.
if req_col
req_row = index
break
end
end
a.each_with_index do |row,index|
row.each_with_index do |col, i|
print ((req_row == index or i == req_col) ? 0 : col).to_s + " "
end
puts "\r"
end
Based on the title of your question, here's solution that searches for positions of the zero values (fixate), then actually zeros out the appropriate row and column (clear, more aligned with the contents of your question):
def fixate matrix, search=0, replace=0
rcs = []
matrix.each_with_index do |row,r|
row.each_with_index do |col,c|
rcs << [ r, c ] if col == search
end
end
rcs.each do |(row, col)|
clear matrix, row, col, replace
end
matrix
end
def clear matrix, row, col, val=0
matrix[row].map! { |_| val } # Clear rows
matrix.each { |r| r[col] = val } # Clear columns
matrix
end
Quick test:
fixate [ # [
[ 1, 2, 3, 4 ], # [ 1, 2, 0, 4 ],
[ 5, 6, 7, 8 ], # [ 5, 6, 0, 8 ],
[ 9, 10, 0, 11 ], # [ 0, 0, 0, 0 ],
[ 12, 13, 14, 15 ] # [ 12, 13, 0, 15 ]
] # ]
Here's what I came up with:
zero_rows=[]
a.map do |col|
col.each_with_index do |el, index|
zero_rows.push(index) if el==0
end
col.fill(0) if col.include?(0)
end
a.map{|col| zero_rows.each{|index| col[index]=0} }
First, use map to iterate through the columns and fill them with zeros if they contain at least one 0. but, while doing so, add the indexes which contain a 0 to the zero_rows array.
Afterwards, iterate through the array once more and set the indexes of each column that match an index in zero_rows to 0.
You may know the map method as collect. They do the same thing.
Side Note:
If an array contains multiple zero's this code will zero out every applicable column. OP's example and some other answers here will only zero out the first column in which a 0 is found. If this is the behavior you are expecting, then see #Doguita's answer.
I don't know if this code is better than the other answers. I will test it later:
ary = [[1, 2, 3, 4], [5, 6, 7, 8], [9, 10, 0, 11], [12, 13, 14, 15]]
col = nil
row = ary.index{ |a| col = a.index(0) }
ary.each_with_index { |a, i| i == row ? a.fill(0) : a[col] = 0 } if col
p ary
# => [[1, 2, 0, 4], [5, 6, 0, 8], [0, 0, 0, 0], [12, 13, 0, 15]]
Obs: This answer assumes there's only one 0 to search in the array