We have an original array and a list of filters where each filter consists of indices which are allowed through the filter. The filters are rather nice, e.g. they are grouped for each power of 2 in the following way (the filters are upto n = 20).
1 (2^0) = 1 3 5 7 9 11 13 15 17 19
2 (2^1) = 1 2 5 6 9 10 13 14 17 18
4 (2^2) = 1 2 3 4 9 10 11 12 17 18 19 20
8 (2^3) = 1 2 3 4 5 6 7 8 17 18 19 20
I hope you get the idea. Now we would apply some or all of these filters (user dictates which filters to apply) to the original array and the xor of the elements of the transformed array is the answer. To take an example if the original array would have been [3 7 8 1 2 9 6 4 11] e.g. n = 9 and we needed to apply the filters of 4, 2 and 1, the transformations would be like this.
After applying filter of 4 - [3 7 8 1 x x x x 11]
After applying filter of 2 - [3 7 x x x x x x 11]
After applying filter of 1 - [3 x x x x x x x 11]
Now the xor of 3 and 11 e.g. 8 is the answer. I can solve this O(n * no. of filters) time, but I need a better solution which might give the answer in O(no of filters) time. Is there any way to take advantage of the properties of xor and/or pre-compute the results for some and then give the answer for the filters. This is because there are many queries with filters, so I need to answer the queries in O(no of filters) time. Any kind of help will be appreciated.
It can be done in O(M) where M is the number of items that pass all filters (independent of the number of filters) by iterating over the array in a particular way, generating only the indexes that pass all the filters.
This is easier to see if you write down the examples starting at zero:
1: 0 2 4 6 8 10 12 14 16 18 (numbers that don't contain 1)
2: 0 1 4 5 8 9 12 13 16 17 (numbers that don't contain 2, etc)
4: 0 1 2 3 8 9 10 11 16 17 18 19
8: 0 1 2 3 4 5 6 7 16 17 18 19
The filters are really just a constraint on the bits of the indexes in the array. That constraint is of the form index & filters = 0, where filters is just the sum of all the individual filters (eg 1 + 2 + 4 = 7). Given a valid index i the next valid index i' can be computed with only primitive operations: i' = (i | filters) + 1 & ~filters. The idea here is to set the bits that are filtered to zero so the +1 will carry through them, then filtered bits are cleared again to make the index valid. The total effect is that the unfiltered bits are incremented and the filtered bits stay zero.
This gives a simple algorithm to iterate directly over all valid indexes. Start at 0 (which is always valid) and increment using the rule above until the end of the array is reached:
for (int i = 0; i < N; i = (i | filters) + 1 & ~filters)
// do something with array[i], like XOR them all together
Related
I am interested in sorting the columns of a matrix in terms of the values in 2 other vectors. As an example, suppose the matrix and vectors look like this:
M = [ 1 2 3 4 5 6 ;
7 8 9 10 11 12 ;
13 14 15 16 17 18 ]
v1 = [ 2 , 6 , 6 , 1 , 3 , 2 ]
v2 = [ 3 , 1 , 2 , 7 , 9 , 1 ]
I want to sort the columns of A in terms of their corresponding values in v1 and v2, with v1 taking precedence over v2. Additionally, I am interested in trying to sort the matrix in place as the matrices I am working with are very large. Currently, my crude solution looks like this:
MM = [ v1' ; v2' ; M ] ; ## concatenate the vectors with the matrix
MM[:,:] = sortcols(MM , by=x->(x[1],x[2]))
M[:,:] = MM[3:end,:]
which gives the desired result:
3x6 Array{Int64,2}:
4 6 1 5 2 3
10 12 7 11 8 9
16 18 13 17 14 15
Clearly my approach is not ideal is it requires computing and storing intermediate matrices. Is there a more efficient/elegant approach for sorting the columns of a matrix in terms of 2 other vectors? And can it be done in place to save memory?
Previously I have used sortperm for sorting an array in terms of the values stored in another vector. Is it possible to use sortperm with 2 vectors (and in-place)?
I would probably do it this way:
julia> cols = sort!([1:size(M,2);], by=i->(v1[i],v2[i]));
julia> M[:,cols]
3×6 Array{Int64,2}:
4 6 1 5 2 3
10 12 7 11 8 9
16 18 13 17 14 15
This should be pretty fast and uses only one temporary vector and one copy of the matrix. It's not fully in-place, but doing this operation completely in-place is not easy. You would need a sorting function that moves columns as it works, or alternatively a version of permute! that works on columns. You could start with the code for permute!! in combinatorics.jl and modify it to permute columns, reusing a single column-size temporary buffer.
I have a matrix of unsorted numbers and I want to keep the n largest (not necessarily unique) values per column and set the rest to zero.
I figured out how to do it with a loop:
a = [4 8 12 5; 9 2 6 18; 11 3 9 7; 8 9 12 4]
k = 2
for n = 1:4
[y, ind] = sort(a(:,n), 'descend');
a(ind(k+1:end),n) = 0;
end
a
which gives me:
a =
4 8 12 5
9 2 6 18
11 3 9 7
8 9 12 4
k =
2
a =
0 8 12 0
9 0 0 18
11 0 0 7
0 9 12 0
However when I try to eliminate the loop, I can't seem to get the indexing right, because this:
a = [4 8 12 5; 9 2 6 18; 11 3 9 7; 8 9 12 4]
k = 2
[y, ind] = sort(a, 'descend');
b = ind(k+1:end,:)
a(b) = 0
which gives me this: (which is not what I wanted to do)
a =
4 8 12 5
9 2 6 18
11 3 9 7
8 9 12 4
k =
2
b =
4 3 3 1
1 2 2 4
a =
0 8 12 5
0 2 6 18
0 3 9 7
0 9 12 4
Am I indexing this wrong? Do I have to use the loop?
I referenced this question to get started but it wasn't exactly what I was trying to do: How to find n largest elements in an array and make the other elements zero in matlab?
You're very close. ind in the sort function gives you the row locations for each column where that particular value would appear in the sorted output. You need to do some additional work if you want to index into the matrix properly and eliminate the entries. You know that for each column of I, that tells you that we need to eliminate those entries from that particular column. Therefore, what I would do is generate column-major linear indices using each column of I to be the rows we need to eliminate.
Try doing this:
a = [4 8 12 5; 9 2 6 18; 11 3 9 7; 8 9 12 4];
k = 2;
[y, ind] = sort(a, 'descend');
%// Change here
b = sub2ind(size(a), ind(k+1:end,:), repmat(1:size(a,2), size(a,1)-k, 1));
a(b) = 0;
We use sub2ind to help us generate our column major indices where the rows are denoted by the values in ind after the kth element and the columns we need are for each column in this matrix. There are size(a,1)-k rows remaining after you truncate out the k values after sorting, and so we generate column values that go from 1 up to as many columns as we have in a and as many rows as there are remaining.
We get this output:
>> a
a =
0 8 12 0
9 0 0 18
11 0 0 7
0 9 12 0
Here's one using bsxfun -
%// Get descending sorting indices per column
[~, ind] = sort(a,1, 'descend')
%// Get linear indices that are to be set to zeros and set those in a to 0s
rem_ind = bsxfun(#plus,ind(n+1:end,:),[0:size(a,2)-1]*size(a,1))
a(rem_ind) = 0
Sample run -
a =
4 8 12 5
9 2 6 18
11 3 9 7
8 9 12 4
n =
2
ind =
3 4 1 2
2 1 4 3
4 3 3 1
1 2 2 4
rem_ind =
4 7 11 13
1 6 10 16
a =
0 8 12 0
9 0 0 18
11 0 0 7
0 9 12 0
I am considering an easy algorithm to rank my 2D array and mark their rank in the same size of the 2D array.
For example, I have a matrix in below:
[0 2 15 34;
0 15 21 24;
0 3 5 8;
1 14 23 29]
The output should be as follow:
[1 5 10 16;
1 10 12 14;
1 6 7 8;
4 9 13 15]
I am kind of new to matlab, I not sure if the matlab have the functionality to directly do it. Or it would be even better if you could provide some ideas for implementing the algorithm. Thank you very much!
If I understand correctly, you want to replace each element by its rank. I offer three ways to do it; the third seems to be what you want.
Let your example data be defined as
data = [0 2 15 34;
0 15 21 24;
0 3 5 8;
1 14 23 29];
This assigns equal ranks to equal data values (as in your example), but doesn't skip ranks in that case (your example seems to do so):
[~, ~, vv] = unique(data(:));
result = reshape(vv, size(data));
With your example data, this gives
result =
1 3 8 13
1 8 9 11
1 4 5 6
2 7 10 12
This assigns different ranks to equal data values (so skipping ranks is out of the question):
[~, vv] = sort(data(:));
[~, vv] = sort(vv);
result = reshape(vv, size(data));
With your example data,
result =
1 5 11 16
2 10 12 14
3 6 7 8
4 9 13 15
This assigns equal ranks to equal data values, and in that case it skips ranks:
[~, vv] = sort(data(:));
[~, vv] = sort(vv);
[~, jj, kk] = unique(data(:), 'first');
result = reshape(vv(jj(kk)), size(data));
With your example data,
result =
1 5 10 16
1 10 12 14
1 6 7 8
4 9 13 15
Another approach, single-line: for each entry, find how many other entries are smaller, and add 1:
result = reshape(sum(bsxfun(#lt,data(:),data(:).'))+1, size(data));
I have a 1 x 15 array of values:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
I need to rearrange them into a 3 x 5 matrix using a for loop:
1 2 3 4 5
6 7 8 9 10
11 12 13 14 15
How would I do that?
I'm going to show you three methods. One where you need to have a for loop, and two others when you don't:
Method #1 - for loop
First, create a matrix that is 3 x 5, then keep track of an index that will go through your array. After, create a double for loop that will help you populate the array.
index = 1;
array = 1 : 15; %// Array we wish to access
matrix = zeros(3,5); %// Initialize
for m = 1 : 3
for n = 1 : 5
matrix(m,n) = array(index);
index = index + 1;
end
end
matrix =
1 2 3 4 5
6 7 8 9 10
11 12 13 14 15
Method #2 - Without a for loop
Simply put, use reshape:
matrix = reshape(1:15, 5, 3).';
matrix =
1 2 3 4 5
6 7 8 9 10
11 12 13 14 15
reshape will take a vector and restructure it into a matrix so that you populate the matrix by columns first. As such, we want to put 1 to 5 in the first column, 6 to 10 in the second and 11 to 15 in the third column. Therefore, our output matrix is in fact 5 x 3. When you see this, this is actually the transposed version of the matrix we want, which is why you do .' to transpose the matrix back.
Method #3 - Another method without a for loop (tip of the hat goes to Luis Mendo)
You can use vec2mat, and specify that you need to have 5 columns worth for your matrix:
matrix = vec2mat(1:15, 5);
matrix =
1 2 3 4 5
6 7 8 9 10
11 12 13 14 15
vec2mat takes a vector and reshapes it into a matrix of as many columns as you specify in the second parameter. In this case, we need 5 columns.
For the sake of (bsx)fun, here is another option...
bsxfun(#plus,1:5,[0:5:10]')
ans =
1 2 3 4 5
6 7 8 9 10
11 12 13 14 15
less readable, maybe faster, but who cares if it is such a small of an array...
A = [ 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 ] ;
A = reshape( A' , 3 , 5 ) ;
A' = 1 2 3 4 5
6 7 8 9 10
11 12 13 14 15
I have an array A, and want to reshape the last four elements of each column into a 2x2 matrix. I would like the results to be stored in a cell array B.
For example, given:
A = [1:6; 3:8; 5:10]';
I would like B to contain three 2x2 arrays, such that:
B{1} = [3, 5; 4, 6];
B{2} = [5, 7; 6, 8];
B{3} = [7, 9; 8, 10];
I can obviously do this in a for loop using something like reshape(A(end-3:end, ii), 2, 2) and looping over ii. Can anyone propose a vectorized method, perhaps using something similar to cellfun that can apply an operation repeatedly to columns of an array?
The way I do this is to look at the desired indices and then figure out a way to generate them, usually using some form of repmat. For example, if you want the last 4 items in each column, the (absolute) indices into A are going to be 3,4,5,6, then add the number of rows to that to move to the next column to get 9,10,11,12 and so on. So the problem becomes generating that matrix in terms of your number of rows, number of columns, and the number of elements you want from each column (I'll call it n, in your case n=4).
octave:1> A = [1:6; 3:8; 5:10]'
A =
1 3 5
2 4 6
3 5 7
4 6 8
5 7 9
6 8 10
octave:2> dim=size(A)
dim =
6 3
octave:3> n=4
n = 4
octave:4> x=repmat((dim(1)-n+1):dim(1),[dim(2),1])'
x =
3 3 3
4 4 4
5 5 5
6 6 6
octave:5> y=repmat((0:(dim(2)-1)),[n,1])
y =
0 1 2
0 1 2
0 1 2
0 1 2
octave:6> ii=x+dim(1)*y
ii =
3 9 15
4 10 16
5 11 17
6 12 18
octave:7> A(ii)
ans =
3 5 7
4 6 8
5 7 9
6 8 10
octave:8> B=reshape(A(ii),sqrt(n),sqrt(n),dim(2))
B =
ans(:,:,1) =
3 5
4 6
ans(:,:,2) =
5 7
6 8
ans(:,:,3) =
7 9
8 10
Depending on how you generate x and y, you can even do away with the multiplication, but I'll leave that to you. :D
IMO you don't need a cell array to store them either, a 3D matrix works just as well and you index into it the same way (but don't forget to squeeze it before you use it).
I gave a similar answer in this question.