Following this question and the precious help I got from it, I've reached to the following issue:
Using indices of detected peaks and having computed the median of my signal +/-3 datapoints around these peaks, I need to replace my signal in a +/-5 window around the peak with the previously computed median.
I'm only able replace the datapoint at the peak with the median, but not the surrounding +/-5 data points...see figure. Black = original peak; Yellow = data point at peak changed to the median of +/-3 datapoints around it.
Original peak and changed peak
Unfortunately I have not been able to make it work by following suggestions on the previous question.
Any help will be very much appreciated!
Cheers,
M
Assuming you mean the following. Given the array
x = [0 1 2 3 4 5 35 5 4 3 2 1 0]
you want to replace 35 and surrounding +/- 5 entries with the median of 3,4,5,35,5,4,3, which is 4, so the resulting array should be
x = [0 4 4 4 4 4 4 4 4 4 4 4 0]
Following my answer in this question an intuitive approach is to simply replace the neighbors with the median value by offsetting the indicies. This can be accomplished as follows
[~,idx]=findpeaks(x);
med_sz = 3; % Take the median with respect to +/- this many neighbors
repl_sz = 5; % Replace neighbors +/- this distance from peak
if ~isempty(idx)
m = medfilt1(x,med_sz*2+1);
N = numel(x);
for offset = -repl_sz:repl_sz
idx_offset = idx + offset;
idx_valid = idx_offset >= 1 & idx_offset <= N;
x(idx_offset(idx_valid)) = m(idx(idx_valid));
end
end
Alternatively, if you want to avoid loops, an equivalent loopless implementation is
[~,idx]=findpeaks(x);
med_sz = 3;
repl_sz = 5;
if ~isempty(idx)
m = medfilt1(x,med_sz*2+1);
idx_repeat = repmat(idx,repl_sz*2+1,1);
idx_offset = idx_repeat + repmat((-repl_sz:repl_sz)',1,numel(idx));
idx_valid = idx_repeat >= 1 & idx_repeat <= numel(x);
idx_repeat = idx_repeat(idx_valid);
idx_offset = idx_offset(idx_valid);
x(idx_offset) = m(idx_repeat);
end
Related
Suppose a data like:
X y
1 5
2 6
3 1
4 7
5 3
6 8
I want to remove 3 1 and 5 3 because their difference with the previous row is more than 3. In fact, I want to draw a plot with them and want it to be smooth.
I tried
for qq = 1:size(data,1)
if data(qq,2) - data(qq-1,2) > 3
data(qq,:)=[];
end
end
However, it gives:
Subscript indices must either be real positive integers or logicals.
Moreover, I guess the size of array changes as I remove some elements.
In the end, the difference between no consecutive elements must be greater than threshold.
In practice I want to smooth the following picture where there is high fluctuate
One very simple filter from Mathematical morphology that you could try is the closing with a structuring element of size 2. It changes the value of any sample that is lower than both neighbors to the lowest of its two neighbors. Other values are not changed. Thus, it doesn't use a threshold to determine what samples are wrong, it only looks that the sample is lower than both neighbors:
y = [5, 6, 1, 7, 3, 8]; % OP's second column
y1 = y;
y1(end+1) = -inf; % enforce boundary condition
y1 = max(y1,circshift(y1,1)); % dilation
y1 = min(y1,circshift(y1,-1)); % erosion
y1 = y1(1:end-1); % undo boundary condition change
This returns y1 = [5 6 6 7 7 8].
If you want to prevent changing your signal for small deviations, you can apply your threshold as a second step:
I = y1 - y < 3;
y1(I) = y(I);
This finds the places where we changed the signal, but the change was less than the threshold of 3. At those places we write back the original value.
You have a few errors:
Your index needs to start from 2, so that you aren't trying to index 0 for a previous index.
You need to check that the absolute value of the difference is greater than 3.
Since your data matrix is changing sizes, you can't use a for loop with a fixed number of iterations. Use a while loop instead.
This should give you the results you want:
qq = 2;
while qq <= size(data, 1)
if abs(data(qq, 2) - data(qq-1, 2)) > 3,
data(qq, :) = [];
else
qq = qq+1;
end
end
I have a vector a = [1 3 4 2 1 5 6 3 2]. Now I want to create a new vector 'b' with the cumsum of a, but after reaching a threshold, let's say 5, cumsum should reset and start again till it reaches the threshold again, so the new vector should look like this:
b = [1 4 4 2 3 5 6 3 5]
Any ideas?
You could build a sparse matrix that, when multiplied by the original vector, returns the cumulative sums. I haven't timed this solution versus others, but I strongly suspect this will be the fastest for large arrays of a.
% Original data
a = [1 3 4 2 1 5 6 3 2];
% Threshold
th = 5;
% Cumulative sum corrected by threshold
b = cumsum(a)/th;
% Group indices to be summed by checking for equality,
% rounded down, between each cumsum value and its next value. We add one to
% prevent NaNs from occuring in the next step.
c = cumsum(floor(b) ~= floor([0,b(1:end-1)]))+1;
% Build the sparse matrix, remove all values that are in the upper
% triangle.
S = tril(sparse(c.'./c == 1));
% In case you use matlab 2016a or older:
% S = tril(sparse(bsxfun(#rdivide,c.',c) == 1));
% Matrix multiplication to create o.
o = S*a.';
By normalizing the arguments of cumsum with the threshold and flooring you can get grouping indizes for accumarray, which then can do the cumsumming groupwise:
t = 5;
a = [1 3 4 2 1 5 6 3 2];
%// cumulative sum of normalized vector a
n = cumsum(a/t);
%// subs for accumarray
subs = floor( n ) + 1;
%// cumsum of every group
aout = accumarray( subs(:), (1:numel(subs)).', [], #(x) {cumsum(a(x))});
%// gather results;
b = [aout{:}]
One way is to use a loop. You create the first cumulative sum cs, and then as long as elements in cs are larger than your threshold th, you replace them with elements from the cumulative sum on the rest of the elements in a.
Because some elements in a might be larger than th, this loop will be infinite unless we also eliminate these elements too.
Here is a simple solution with a while loop:
a = [1 3 4 2 1 5 6 3 2];
th = 5;
cs = cumsum(a);
while any(cs>th & cs~=a) % if 'cs' has values larger that 'th',
% and there are any values smaller than th left in 'a'
% sum all the values in 'a' that are after 'cs' reached 'th',
% excluding values that are larger then 'th'
cs(cs>th & cs~=a) = cumsum(a(cs>th & cs~=a));
end
Calculate the cumulative sum and replace the indices value obeying your condition.
a = [1 3 4 2 1 5 6 3 2] ;
b = [1 4 4 2 3 5 6 3 5] ;
iwant = a ;
a_sum = cumsum(a) ;
iwant(a_sum<5) = a_sum(a_sum<5) ;
I have a matlab problem to solve. In have two vectores that limit my space, x_low and x_high. The matrix pos needs to have values within this spaces and each column of the matrix has different bounds given by the two vectores. Now my problem is that randi gives valus between two integers but i need to change the bounds for each columns. There is another way to use randi or a different matlab function to do this?
I know there are better codes to do this but i'm starting to use matlab and i know to do it this way, any aid is welcome
x_low = [Io_low, Iirr_low, Rp_low, Rs_low, n_low]; % vector of constant values
x_high = [Io_high, Iirr_high, Rp_high, Rs_high, n_high]; % vector of constant values
pos = rand(particles, var);
var = length(x_high);
for i = 1: particles % rows
for k = 1: var %columns
if pos(i, k) < x_low(k) || pos(i, k) > x_high(k) % if the position is out of bounder
pos(i, k) = randi(x_low(k), x_high(k), 1); % fill it with a particle whithin the bounderies
end
end
end
If I understand correctly, you need to generate a matrix with integer values such that each column has different lower and upper limits; and those lower and upper limits are inclusive.
This can be done very simply with
rand (to generate random numbers between 0 and 1 ),
bsxfun (to take care of the lower and upper limits on a column basis), and
round (so that the results are integer values).
Let the input data be defined as
x_low = [1 6 11]; %// lower limits
x_high = [3 10 100]; %// upper limits
n_rows = 7; %// number of columns
Then:
r = rand(n_rows, numel(x_low)); %// random numbers between 0 and 1
r = floor(bsxfun(#times, r, x_high-x_low+1)); %// adjust span and round to integers
r = bsxfun(#plus, r, x_low); %// adjust lower limit
gives something like
r =
2 7 83
3 6 93
2 6 22
3 10 85
3 7 96
1 10 90
2 8 57
If you need to fill in values only at specific entries of matrix pos, you can use something like
ind = bsxfun(#lt, pos, x_low) | bsxfun(#gt, pos, x_high); %// index of values to replace
pos(ind) = r(ind);
This a little overkill, because the whole matrixd r is generated only to use some of its entries. To generate only the needed values the best way is probably to use loops.
You can use cellfun for this. Something like:
x_low = [Io_low, Iirr_low, Rp_low, Rs_low, n_low];
x_high = [Io_high, Iirr_high, Rp_high, Rs_high, n_high];
pos = cell2mat(cellfun(#randi, mat2cell([x_low' x_high'], ones(numel(x_low),1), 1), repmat({[particles 1]}, [numel(x_low) 1)])))';
Best,
I'm trying to elegantly split a vector. For example,
vec = [1 2 3 4 5 6 7 8 9 10]
According to another vector of 0's and 1's of the same length where the 1's indicate where the vector should be split - or rather cut:
cut = [0 0 0 1 0 0 0 0 1 0]
Giving us a cell output similar to the following:
[1 2 3] [5 6 7 8] [10]
Solution code
You can use cumsum & accumarray for an efficient solution -
%// Create ID/labels for use with accumarray later on
id = cumsum(cut)+1
%// Mask to get valid values from cut and vec corresponding to ones in cut
mask = cut==0
%// Finally get the output with accumarray using masked IDs and vec values
out = accumarray(id(mask).',vec(mask).',[],#(x) {x})
Benchmarking
Here are some performance numbers when using a large input on the three most popular approaches listed to solve this problem -
N = 100000; %// Input Datasize
vec = randi(100,1,N); %// Random inputs
cut = randi(2,1,N)-1;
disp('-------------------- With CUMSUM + ACCUMARRAY')
tic
id = cumsum(cut)+1;
mask = cut==0;
out = accumarray(id(mask).',vec(mask).',[],#(x) {x});
toc
disp('-------------------- With FIND + ARRAYFUN')
tic
N = numel(vec);
ind = find(cut);
ind_before = [ind-1 N]; ind_before(ind_before < 1) = 1;
ind_after = [1 ind+1]; ind_after(ind_after > N) = N;
out = arrayfun(#(x,y) vec(x:y), ind_after, ind_before, 'uni', 0);
toc
disp('-------------------- With CUMSUM + ARRAYFUN')
tic
cutsum = cumsum(cut);
cutsum(cut == 1) = NaN; %Don't include the cut indices themselves
sumvals = unique(cutsum); % Find the values to use in indexing vec for the output
sumvals(isnan(sumvals)) = []; %Remove NaN values from sumvals
output = arrayfun(#(val) vec(cutsum == val), sumvals, 'UniformOutput', 0);
toc
Runtimes
-------------------- With CUMSUM + ACCUMARRAY
Elapsed time is 0.068102 seconds.
-------------------- With FIND + ARRAYFUN
Elapsed time is 0.117953 seconds.
-------------------- With CUMSUM + ARRAYFUN
Elapsed time is 12.560973 seconds.
Special case scenario: In cases where you might have runs of 1's, you need to modify few things as listed next -
%// Mask to get valid values from cut and vec corresponding to ones in cut
mask = cut==0
%// Setup IDs differently this time. The idea is to have successive IDs.
id = cumsum(cut)+1
[~,~,id] = unique(id(mask))
%// Finally get the output with accumarray using masked IDs and vec values
out = accumarray(id(:),vec(mask).',[],#(x) {x})
Sample run with such a case -
>> vec
vec =
1 2 3 4 5 6 7 8 9 10
>> cut
cut =
1 0 0 1 1 0 0 0 1 0
>> celldisp(out)
out{1} =
2
3
out{2} =
6
7
8
out{3} =
10
For this problem, a handy function is cumsum, which can create a cumulative sum of the cut array. The code that produces an output cell array is as follows:
vec = [1 2 3 4 5 6 7 8 9 10];
cut = [0 0 0 1 0 0 0 0 1 0];
cutsum = cumsum(cut);
cutsum(cut == 1) = NaN; %Don't include the cut indices themselves
sumvals = unique(cutsum); % Find the values to use in indexing vec for the output
sumvals(isnan(sumvals)) = []; %Remove NaN values from sumvals
output = {};
for i=1:numel(sumvals)
output{i} = vec(cutsum == sumvals(i)); %#ok<SAGROW>
end
As another answer shows, you can use arrayfun to create a cell array with the results. To apply that here, you'd replace the for loop (and the initialization of output) with the following line:
output = arrayfun(#(val) vec(cutsum == val), sumvals, 'UniformOutput', 0);
That's nice because it doesn't end up growing the output cell array.
The key feature of this routine is the variable cutsum, which ends up looking like this:
cutsum =
0 0 0 NaN 1 1 1 1 NaN 2
Then all we need to do is use it to create indices to pull the data out of the original vec array. We loop from zero to max and pull matching values. Notice that this routine handles some situations that may arise. For instance, it handles 1 values at the very beginning and very end of the cut array, and it gracefully handles repeated ones in the cut array without creating empty arrays in the output. This is because of the use of unique to create the set of values to search for in cutsum, and the fact that we throw out the NaN values in the sumvals array.
You could use -1 instead of NaN as the signal flag for the cut locations to not use, but I like NaN for readability. The -1 value would probably be more efficient, as all you'd have to do is truncate the first element from the sumvals array. It's just my preference to use NaN as a signal flag.
The output of this is a cell array with the results:
output{1} =
1 2 3
output{2} =
5 6 7 8
output{3} =
10
There are some odd conditions we need to handle. Consider the situation:
vec = [1 2 3 4 5 6 7 8 9 10 11 12 13 14];
cut = [1 0 0 1 1 0 0 0 0 1 0 0 0 1];
There are repeated 1's in there, as well as a 1 at the beginning and end. This routine properly handles all this without any empty sets:
output{1} =
2 3
output{2} =
6 7 8 9
output{3} =
11 12 13
You can do this with a combination of find and arrayfun:
vec = [1 2 3 4 5 6 7 8 9 10];
N = numel(vec);
cut = [0 0 0 1 0 0 0 0 1 0];
ind = find(cut);
ind_before = [ind-1 N]; ind_before(ind_before < 1) = 1;
ind_after = [1 ind+1]; ind_after(ind_after > N) = N;
out = arrayfun(#(x,y) vec(x:y), ind_after, ind_before, 'uni', 0);
We thus get:
>> celldisp(out)
out{1} =
1 2 3
out{2} =
5 6 7 8
out{3} =
10
So how does this work? Well, the first line defines your input vector, the second line finds how many elements are in this vector and the third line denotes your cut vector which defines where we need to cut in our vector. Next, we use find to determine the locations that are non-zero in cut which correspond to the split points in the vector. If you notice, the split points determine where we need to stop collecting elements and begin collecting elements.
However, we need to account for the beginning of the vector as well as the end. ind_after tells us the locations of where we need to start collecting values and ind_before tells us the locations of where we need to stop collecting values. To calculate these starting and ending positions, you simply take the result of find and add and subtract 1 respectively.
Each corresponding position in ind_after and ind_before tell us where we need to start and stop collecting values together. In order to accommodate for the beginning of the vector, ind_after needs to have the index of 1 inserted at the beginning because index 1 is where we should start collecting values at the beginning. Similarly, N needs to be inserted at the end of ind_before because this is where we need to stop collecting values at the end of the array.
Now for ind_after and ind_before, there is a degenerate case where the cut point may be at the end or beginning of the vector. If this is the case, then subtracting or adding by 1 will generate a start and stopping position that's out of bounds. We check for this in the 4th and 5th line of code and simply set these to 1 or N depending on whether we're at the beginning or end of the array.
The last line of code uses arrayfun and iterates through each pair of ind_after and ind_before to slice into our vector. Each result is placed into a cell array, and our output follows.
We can check for the degenerate case by placing a 1 at the beginning and end of cut and some values in between:
vec = [1 2 3 4 5 6 7 8 9 10];
cut = [1 0 0 1 0 0 0 1 0 1];
Using this example and the above code, we get:
>> celldisp(out)
out{1} =
1
out{2} =
2 3
out{3} =
5 6 7
out{4} =
9
out{5} =
10
Yet another way, but this time without any loops or accumulating at all...
lengths = diff(find([1 cut 1])) - 1; % assuming a row vector
lengths = lengths(lengths > 0);
data = vec(~cut);
result = mat2cell(data, 1, lengths); % also assuming a row vector
The diff(find(...)) construct gives us the distance from each marker to the next - we append boundary markers with [1 cut 1] to catch any runs of zeros which touch the ends. Each length is inclusive of its marker, though, so we subtract 1 to account for that, and remove any which just cover consecutive markers, so that we won't get any undesired empty cells in the output.
For the data, we mask out any elements corresponding to markers, so we just have the valid parts we want to partition up. Finally, with the data ready to split and the lengths into which to split it, that's precisely what mat2cell is for.
Also, using #Divakar's benchmark code;
-------------------- With CUMSUM + ACCUMARRAY
Elapsed time is 0.272810 seconds.
-------------------- With FIND + ARRAYFUN
Elapsed time is 0.436276 seconds.
-------------------- With CUMSUM + ARRAYFUN
Elapsed time is 17.112259 seconds.
-------------------- With mat2cell
Elapsed time is 0.084207 seconds.
...just sayin' ;)
Here's what you need:
function spl = Splitting(vec,cut)
n=1;
j=1;
for i=1:1:length(b)
if cut(i)==0
spl{n}(j)=vec(i);
j=j+1;
else
n=n+1;
j=1;
end
end
end
Despite how simple my method is, it's in 2nd place for performance:
-------------------- With CUMSUM + ACCUMARRAY
Elapsed time is 0.264428 seconds.
-------------------- With FIND + ARRAYFUN
Elapsed time is 0.407963 seconds.
-------------------- With CUMSUM + ARRAYFUN
Elapsed time is 18.337940 seconds.
-------------------- SIMPLE
Elapsed time is 0.271942 seconds.
Unfortunately there is no 'inverse concatenate' in MATLAB. If you wish to solve a question like this you can try the below code. It will give you what you looking for in the case where you have two split point to produce three vectors at the end. If you want more splits you will need to modify the code after the loop.
The results are in n vector form. To make them into cells, use num2cell on the results.
pos_of_one = 0;
% The loop finds the split points and puts their positions into a vector.
for kk = 1 : length(cut)
if cut(1,kk) == 1
pos_of_one = pos_of_one + 1;
A(1,one_pos) = kk;
end
end
F = vec(1 : A(1,1) - 1);
G = vec(A(1,1) + 1 : A(1,2) - 1);
H = vec(A(1,2) + 1 : end);
I am trying to allocate (x, y) points to the cells of a non-uniform rectangular grid. Simply speaking, I have a grid defined as a sorted non-equidistant array
xGrid = [x1, x2, x3, x4];
and an array of numbers x lying between x1 and x4. For each x, I want to find its position in xGrid, i.e. such i that
xGrid(i) <= xi <= xGrid(i+1)
Is there a better (faster/simpler) way to do it than arrayfun(#(x) find(xGrid <= x, 1, 'last'), x)?
You are looking for the second output of histc:
[~,where] = histc(x, xGrid)
This returns the array where such that xGrid(where(i)) <= x(i) < xGrid(where(i)+1) holds.
Example:
xGrid = [2,4,6,8,10];
x = [3,5,6,9,11];
[~,where] = histc(x, xGrid)
Yields the following output:
where =
1 2 3 4 0
If you want xGrid(where(i)) < x(i) <= xGrid(where(i)+1), you need to do some trickery of negating the values:
[~,where] = histc(-x,-flip(xGrid));
where(where~=0) = numel(xGrid)-where(where~=0)
This yields:
where =
1 2 2 4 0
Because x(3)==6 is now counted for the second interval (4,6] instead of [6,8) as before.
Using bsxfun for the comparisons and exploiting find-like capabilities of max's second output:
xGrid = [2 4 6 8]; %// example data
x = [3 6 5.5 10 -10]; %// example data
comp = bsxfun(#gt, xGrid(:), x(:).'); %'// see if "x" > "xGrid"
[~, result] = max(comp, [], 1); %// index of first "xGrid" that exceeds each "x"
result = result-1; %// subtract 1 to find the last "xGrid" that is <= "x"
This approach gives 0 for values of x that lie outside xGrid. With the above example values,
result =
1 3 2 0 0
See if this works for you -
matches = bsxfun(#le,xGrid(1:end-1),x(:)) & bsxfun(#ge,xGrid(2:end),x(:))
[valid,pos] = max(cumsum(matches,2),[],2)
pos = pos.*(valid~=0)
Sample run -
xGrid =
5 2 1 6 8 9 2 1 6
x =
3 7 14
pos =
8
4
0
Explanation on the sample run -
First element of x, 3 occurs last between ...1 6 with the criteria of xGrid(i) <= xi <= xGrid(i+1) at the backend of xGrid and that 1 is at the eight position, so the first element of the output pos is 8. This continues for the second element 7, which is found between 6 and 8 and that 6 is at the fourth place in xGrid, so the second element of the output is 4. For the third element 14 which doesn't find any neighbours to satisfy the criteria xGrid(i) <= xi <= xGrid(i+1) and is therefore outputted as 0.
If x is a column this might help
xg1=meshgrid(xGrid,1:length(x));
xg2=ndgrid(x,1:length(xGrid));
[~,I]=min(floor(abs(xg1-xg2)),[],2);
or a single line implementation
[~,I]=min(floor(abs(meshgrid(xGrid,1:length(x))-ndgrid(x,1:length(xGrid)))),[],2);
Example: xGrid=[1 2 3 4 5], x=[2.5; 1.3; 1.7; 4.8; 3.3]
Result:
I =
2
1
1
4
3