I basically want to use the function ismember, but for a range. For example, I want to know what data points in array1 are within n distance to array2, for each element in array2.
I have the following:
array1 = [1,2,3,4,5]
array2 = [2,2,3,10,20,40,50]
I want to know what values in array2 are <= 2 away from array1:
indices(1,:) (where array1(1) = 1) = [1 1 1 0 0 0 0]
indices(2,:) (where array1(2) = 2) = [1 1 1 0 0 0 0]
indices(3,:) (where array1(3) = 3) = [1 1 1 0 0 0 0]
indices(4,:) (where array1(4) = 4) = [1 1 1 0 0 0 0]
indices(5,:) (where array1(5) = 5) = [0 0 1 0 0 0 0]
Drawbacks:
My array1 is 496736 elements, my array2 is 9268 elements, so aI would rather not use a loop.
Looping is a valid option here. Intialise an array output to the size of array1 X array2, then loop over all elements in array1 an subtract array2 from that, then check whether the absolute value is less than or equal to 2:
array1 = [1,2,3,4,5];
array2 = [2,2,3,10,20,40,50];
output = zeros(numel(array1), numel(array2),'logical');
for ii = 1:numel(array1)
output(ii,:) = abs(array1(ii)-array2)<=2;
end
output =
1 1 1 0 0 0 0
1 1 1 0 0 0 0
1 1 1 0 0 0 0
1 1 1 0 0 0 0
0 0 1 0 0 0 0
i.e. loops are not the problem.
Thanks to Rahnema1's suggestion, you can initialise output directly as a logical matrix:
output = zeros(numel(array1),numel(array2),'logical');
whose size is just 4.3GB.
On timings: Hans' code runs in a matter of seconds for array1 = 5*rand(496736,1); array2 = 25*rand(9286,1);, the looped solution takes about 15 times longer. Both solutions are equal to one another. obcahrdon's ismembertol solution is somewhere in between on my machine.
On RAM usage:
Both implicit expansion, as per Hans' answer, as well as the loop suggested in mine work with just 4.3GB RAM on your expanded problem size (496736*9286)
pdist2 as per Luis' answer and bsxfun as per Hans' on the other hand try to create an intermediate double matrix of 34GB (which doesn't even fit in my RAM, so I cannot compare timings).
obchardon's ismembertol solution outputs a different form of the solution, and takes ~5.04GB (highly dependent on the amount of matches found, the more, the larger this number will be).
In general this leads me to the conclusion that implicit expansion should be your option of choice, but if you have R2016a or earlier, ismembertol or a loop is the way to go.
Using implicit expansion, introduced in MATLAB R2016b, you can simply write:
abs(array1.' - array2) <= 2
ans =
1 1 1 0 0 0 0
1 1 1 0 0 0 0
1 1 1 0 0 0 0
1 1 1 0 0 0 0
0 0 1 0 0 0 0
For earlier MATLAB versions, you can get this using the bsxfun function:
abs(bsxfun(#minus, array1.', array2)) <= 2
ans =
1 1 1 0 0 0 0
1 1 1 0 0 0 0
1 1 1 0 0 0 0
1 1 1 0 0 0 0
0 0 1 0 0 0 0
Hope that helps!
P.S. On the "MATLAB is slow for loops" myth, please have a look at that blog post for example.
EDIT: Please read Adriaan's answer on the RAM consumption using this and/or his approach!
If you have the Statistics Toolbox, you can use pdist2 to compute the distance matrix:
result = pdist2(array1(:), array2(:)) <= 2;
As noted by #Adriaan, this is not efficient for large input sizes. In that case a better approach is a loop with preallocation of the logical matrix output, as in his answer.
You can also use ismembertol with some specific option:
A = [1,2,3,4,5];
B = [2,2,3,10,20,40,5000];
tol = 2;
[~,ind] = ismembertol([A-tol;A+tol].',[B-tol;B+tol].',tol, 'ByRows', true, ...
'OutputAllIndices', true, 'DataScale', [1,Inf])
It will create a 5x1 cell array containing the corresponding linear indice
ind{1} = [1,2,3]
ind{2} = [1,2,3]
...
ind{5} = [3]
In this case using linear indices instead of logical indices will greatly reduce the memory consumption.
Related
I have an array A of 1s and 0s and want to see if the larger array of bits B contains those bits in that exact order?
Example: A= [0 1 1 0 0 0 0 1]
B= [0 1 0 0 1 1 0 0 0 0 1 0 1 0 1]
would be true as A is contained in B
Most solutions I have found only determine if a value IS contained in another matrix, this is no good here as it is already certain that both matrices will be 1s and 0s
Thanks
One (albeit unusual) option, since you're dealing with integer values, is to convert A and B to character arrays and use the contains function:
isWithin = contains(char(B), char(A));
There are some obtuse vectorized ways to to do this, but by far the easiest, and likely just as efficient, is to use a loop with a sliding window,
A = [0 1 1 0 0 0 0 1];
B = [0 1 0 0 1 1 0 0 0 0 1 0 1 0 1];
vec = 0:(numel(A)-1);
for idx = 1:(numel(B)-numel(A)-1)
if all(A==B(idx+vec))
fprintf('A is contained in B\n');
break; % exit the loop as soon as 1 match is found
end
end
Or if you want to know the location(s) in B (of potentially multiple matches) then,
A = [0 1 1 0 0 0 0 1];
B = [0 1 0 0 1 1 0 0 0 0 1 0 1 0 1];
C = false(1,numel(B)-numel(A)-1);
vec = 0:(numel(A)-1);
for idx = 1:numel(C)
C(idx) = all(A==B(idx+vec));
end
if any(C)
fprintf('A is contained in B\n');
end
In this case
>> C
C =
1×6 logical array
0 0 0 1 0 0
You can use the cross-correlation between two signals for this, as a measure of local similarity.
For achieving good results, you need to shift A and B so that you don't have the value 0 any more. Then compute the correlation between the two of them with conv (keeping in mind that the convolution is the cross-correlation with one signal flipped), and normalize with the energy of A so that you get a perfect match whenever you get the value 1:
conv(B-0.5, flip(A)-0.5, 'valid')/sum((A-0.5).^2)
In the normalization term, flipping is removed as it does not change the value.
It gives:
[0 -0.5 0.25 1 0 0 -0.25 0]
4th element is 1, so starting from index equal to 4 you get a perfect match.
I have a matrix
a =
0 0 0 0 0
0 0 0 0 0
0 0 0 0 0
0 0 0 0 0
0 0 0 0 0
0 0 0 0 0
and b vector
b =
1 2 3 4 5 5
I want to replace value of each row in a matrix with reference value of b matrix value and finally generate a matrix as follows without using for loop.
a_new =
1 0 0 0 0
0 1 0 0 0
0 0 1 0 0
0 0 0 1 0
0 0 0 0 1
0 0 0 0 1
if first element of b, b(1) = 1 so change take first row of a vector and make first element as 1 because b(1) = 1.
How can I implement this without using for loop?
Sure. You only need to build a linear index from b and use it to fill the values in a:
a = zeros(6,5); % original matrix
b = [1 2 3 4 5 5]; % row or column vector with column indices into a
ind = (1:size(a,1)) + (b(:).'-1)*size(a,1); % build linear index
a(ind) = 1; % fill value at those positions
Same as Luis Mendo's answer, but using the dedicated function sub2ind:
a( sub2ind(size(a),(1:numel(b)).',b(:)) ) = 1
Also via the subscript to indices conversion way,
a = zeros(6,5);
b = [1 2 3 4 5 5];
idx = sub2ind(size(a), [1:6], b); % 1:6 just to create the row index per b entry
a(idx) = 1
any of these methods works in Octave:
bsxfun(#eq, [1:5 5]',(1:5))
[1:5 5].' == (1:5)
I have an array (say of 1s and 0s) and I want to find the index, i, for the first location where 1 appears n times in a row.
For example,
x = [0 0 1 0 1 1 1 0 0 0] ;
i = 5, for n = 3, as this is the first time '1' appears three times in a row.
Note: I want to find where 1 appears n times in a row so
i = find(x,n,'first');
is incorrect as this would give me the index of the first n 1s.
It is essentially a string search? eg findstr but with a vector.
You can do it with convolution as follows:
x = [0 0 1 0 1 1 1 0 0 0];
N = 3;
result = find(conv(x, ones(1,N), 'valid')==N, 1)
How it works
Convolve x with a vector of N ones and find the first time the result equals N. Convolution is computed with the 'valid' flag to avoid edge effects and thus obtain the correct value for the index.
Another answer that I have is to generate a buffer matrix where each row of this matrix is a neighbourhood of overlapping n elements of the array. Once you create this, index into your array and find the first row that has all 1s:
x = [0 0 1 0 1 1 1 0 0 0]; %// Example data
n = 3; %// How many times we look for duplication
%// Solution
ind = bsxfun(#plus, (1:numel(x)-n+1).', 0:n-1); %'
out = find(all(x(ind),2), 1);
The first line is a bit tricky. We use bsxfun to generate a matrix of size m x n where m is the total number of overlapping neighbourhoods while n is the size of the window you are searching for. This generates a matrix where the first row is enumerated from 1 to n, the second row is enumerated from 2 to n+1, up until the very end which is from numel(x)-n+1 to numel(x). Given n = 3, we have:
>> ind
ind =
1 2 3
2 3 4
3 4 5
4 5 6
5 6 7
6 7 8
7 8 9
8 9 10
These are indices which we will use to index into our array x, and for your example it generates the following buffer matrix when we directly index into x:
>> x = [0 0 1 0 1 1 1 0 0 0];
>> x(ind)
ans =
0 0 1
0 1 0
1 0 1
0 1 1
1 1 1
1 1 0
1 0 0
0 0 0
Each row is an overlapping neighbourhood of n elements. We finally end by searching for the first row that gives us all 1s. This is done by using all and searching over every row independently with the 2 as the second parameter. all produces true if every element in a row is non-zero, or 1 in our case. We then combine with find to determine the first non-zero location that satisfies this constraint... and so:
>> out = find(all(x(ind), 2), 1)
out =
5
This tells us that the fifth location of x is where the beginning of this duplication occurs n times.
Based on Rayryeng's approach you can loop this as well. This will definitely be slower for short array sizes, but for very large array sizes this doesn't calculate every possibility, but stops as soon as the first match is found and thus will be faster. You could even use an if statement based on the initial array length to choose whether to use the bsxfun or the for loop. Note also that for loops are rather fast since the latest MATLAB engine update.
x = [0 0 1 0 1 1 1 0 0 0]; %// Example data
n = 3; %// How many times we look for duplication
for idx = 1:numel(x)-n
if all(x(idx:idx+n-1))
break
end
end
Additionally, this can be used to find the a first occurrences:
x = [0 0 1 0 1 1 1 0 0 0 0 0 1 0 1 1 1 0 0 0 0 0 1 0 1 1 1 0 0 0]; %// Example data
n = 3; %// How many times we look for duplication
a = 2; %// number of desired matches
collect(1,a)=0; %// initialise output
kk = 1; %// initialise counter
for idx = 1:numel(x)-n
if all(x(idx:idx+n-1))
collect(kk) = idx;
if kk == a
break
end
kk = kk+1;
end
end
Which does the same but shuts down after a matches have been found. Again, this approach is only useful if your array is large.
Seeing you commented whether you can find the last occurrence: yes. Same trick as before, just run the loop backwards:
for idx = numel(x)-n:-1:1
if all(x(idx:idx+n-1))
break
end
end
One possibility with looping:
i = 0;
n = 3;
for idx = n : length(x)
idx_true = 1;
for sub_idx = (idx - n + 1) : idx
idx_true = idx_true & (x(sub_idx));
end
if(idx_true)
i = idx - n + 1;
break
end
end
if (i == 0)
disp('No index found.')
else
disp(i)
end
I am trying to write a svm code, but i am literally a beginner in matlab.
So in my code, in a for loop, i should store predictions. The data is like this:
testIdx = [1 1 1 0 0 0 0 0 1 0 1 1]'; % i wrote it like this but it says logical
and
pred = [1 1 1 0 1 0]'; % again logical
So i want to form a 12 length array and turn its 1st,2nd,3rd,9th,11th,12th elements into 1 1 1 0 1 0, and likewise rest of test elements into another set of 0/1s in other iteration.
If possible let it be a normal array, not logical. Thanks in advance
I did it myself old style but there must be a shorter direct way right?
Y = zeros ( size(testIdx,1), 1) ;
a=1;
for i = 1:size(testIdx,1)
if testIdx(i) ==1
Y(i) = pred(a);
a=a+1;
end
end
If you create testIdx and pred the way you specified, then they are double and not logical type. To use logical indexing, it is easiest if testIdx is converted to the logical type. Then you can simply use
testIdx = [1 1 1 0 0 0 0 0 1 0 1 1]';
pred = [1 1 1 0 1 0]';
Y = zeros(size(testIdx));
Y(logical(testIdx)) = pred;
With Y(logical(testIdx), you select all indexes which are set to 1 in the testIdx vector and then write predto these indexes.
Given a 1*N matrix or an array, how do I find the first 4 elements which have the same value and then store the index for those elements?
PS:
I'm just curious. What if we want to find the first 4 elements whose value differences are within a certain range, say below 2? For example, M=[10,15,14.5,9,15.1,8.5,15.5,9.5], the elements I'm looking for will be 15,14.5,15.1,15.5 and the indices will be 2,3,5,7.
If you want the first value present 4 times in the array 'tab' in Matlab, you can use
num_min = 4
val=NaN;
for i = tab
if sum(tab==i) >= num_min
val = i;
break
end
end
ind = find(tab==val, num_min);
By instance with
tab = [2 4 4 5 4 6 4 5 5 4 6 9 5 5]
you get
val =
4
ind =
2 3 5 7
Here is my MATLAB solution:
array = randi(5, [1 10]); %# random array of integers
n = unique(array)'; %'# unique elements
[r,~] = find(cumsum(bsxfun(#eq,array,n),2) == 4, 1, 'first');
if isempty(r)
val = []; ind = []; %# no answer
else
val = n(r); %# the value found
ind = find(array == val, 4); %# indices of elements corresponding to val
end
Example:
array =
1 5 3 3 1 5 4 2 3 3
val =
3
ind =
3 4 9 10
Explanation:
First of all, we extract the list of unique elements. In the example used above, we have:
n =
1
2
3
4
5
Then using the BSXFUN function, we compare each unique value against the entire vector array we have. This is equivalent to the following:
result = zeros(length(n),length(array));
for i=1:length(n)
result(i,:) = (array == n(i)); %# row-by-row
end
Continuing with the same example we get:
result =
1 0 0 0 1 0 0 0 0 0
0 0 0 0 0 0 0 1 0 0
0 0 1 1 0 0 0 0 1 1
0 0 0 0 0 0 1 0 0 0
0 1 0 0 0 1 0 0 0 0
Next we call CUMSUM on the result matrix to compute the cumulative sum along the rows. Each row will give us how many times the element in question appeared so far:
>> cumsum(result,2)
ans =
1 1 1 1 2 2 2 2 2 2
0 0 0 0 0 0 0 1 1 1
0 0 1 2 2 2 2 2 3 4
0 0 0 0 0 0 1 1 1 1
0 1 1 1 1 2 2 2 2 2
Then we compare that against four cumsum(result,2)==4 (since we want the location where an element appeared for the forth time):
>> cumsum(result,2)==4
ans =
0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 1
0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
Finally we call FIND to look for the first appearing 1 according to a column-wise order: if we traverse the matrix from the previous step column-by-column, then the row of the first appearing 1 indicates the index of the element we are looking for. In this case, it was the third row (r=3), thus the third element in the unique vector is the answer val = n(r). Note that if we had multiple elements repeated 4 times or more in the original array, then the one first appearing for the forth time will show up first as a 1 going column-by-column in the above expression.
Finding the indices of the corresponding answer value is a simple call to FIND...
Here is C++ code
std::map<int,std::vector<int> > dict;
std::vector<int> ans(4);//here we will store indexes
bool noanswer=true;
//my_vector is a vector, which we must analize
for(int i=0;i<my_vector.size();++i)
{
std::vector<int> &temp = dict[my_vector[i]];
temp.push_back(i);
if(temp.size()==4)//we find ans
{
std::copy(temp.begin(),temp.end(),ans.begin() );
noanswer = false;
break;
}
}
if(noanswer)
std::cout<<"No Answer!"<<std::endl;
Ignore this and use Amro's mighty solution . . .
Here is how I'd do it in Matlab. The matrix can be any size and contain any range of values and this should work. This solution will automatically find a value and then the indicies of the first 4 elements without being fed the search value a priori.
tab = [2 5 4 5 4 6 4 5 5 4 6 9 5 5]
%this is a loop to find the indicies of groups of 4 identical elements
tot = zeros(size(tab));
for nn = 1:numel(tab)
idxs=find(tab == tab(nn), 4, 'first');
if numel(idxs)<4
tot(nn) = Inf;
else
tot(nn) = sum(idxs);
end
end
%find the first 4 identical
bestTot = find(tot == min(tot), 1, 'first' );
%store the indicies you are interested in.
indiciesOfInterst = find(tab == tab(bestTot), 4, 'first')
Since I couldn't easily understand some of the solutions, I made that one:
l = 10; m = 5; array = randi(m, [1 l])
A = zeros(l,m); % m is the maximum value (may) in array
A(sub2ind([l,m],1:l,array)) = 1;
s = sum(A,1);
b = find(s(array) == 4,1);
% now in b is the index of the first element
if (~isempty(b))
find(array == array(b))
else
disp('nothing found');
end
I find this easier to visualize. It fills '1' in all places of a square matrix, where values in array exist - according to their position (row) and value (column). This is than summed up easily and mapped to the original array. Drawback: if array contains very large values, A may get relative large too.
You're PS question is more complicated. I didn't have time to check each case but the idea is here :
M=[10,15,14.5,9,15.1,8.5,15.5,9.5]
val = NaN;
num_min = 4;
delta = 2;
[Ms, iMs] = sort(M);
dMs = diff(Ms);
ind_min=Inf;
n = 0;
for i = 1:length(dMs)
if dMs(i) <= delta
n=n+1;
else
n=0;
end
if n == (num_min-1)
if (iMs(i) < ind_min)
ind_min = iMs(i);
end
end
end
ind = sort(iMs(ind_min + (0:num_min-1)))
val = M(ind)