The problem: I have roughly isolated data points of the baselines either side of a square pulse signal. I now want to clip these arrays of data points from the 'pulse end' to their respective means (because I still have some data points from the rise and fall of the signal).
As you can see my method appears to work when starting at the end of an array and clipping backwards, but seems to skip data points when assessing array indexes starting from the beginning of the array.
The code:
baseline1=[1,2,1,2,1,2,1,2,1,2,1,2,1,2,1,2,3,4;1:18];
baseline2=[4,3,2,1,2,1,2,1,2,1,2,1,2,1,2,1,2,1;32:49];
%---------------------------------------------------
figure()
hold on
plot(baseline1(2,:),baseline1(1,:),'k')
plot(baseline2(2,:),baseline2(1,:),'k')
%---------------------------------------------------
%Clip baseline1
arrayMean = mean(baseline1(1,:));
x=size(baseline1,2);
meanCrossed = 0;
arrayClipped = baseline1;
for i=x:-1:1
if baseline1(1,i)>arrayMean && meanCrossed == 0
arrayClipped(:,i)=[];
else
meanCrossed = 1;
end
end
baseline1=arrayClipped;
%---------------------------------------------------
%Clip baseline2
arrayMean = mean(baseline2(1,:));
x=size(baseline2,2);
meanCrossed = 0;
arrayClipped = baseline2;
for i=1:x
if baseline2(1,i)>arrayMean && meanCrossed == 0
arrayClipped(:,i)=[];
else
meanCrossed = 1;
end
end
baseline2=arrayClipped;
%---------------------------------------------------
plot(baseline1(2,:),baseline1(1,:),'g','LineWidth',2)
plot(baseline2(2,:),baseline2(1,:),'g','LineWidth',2)
Can anyone advise?
Say you want to delete indices 4 then 2 from a vector of size 4. 4 goes away, so now it's size 3. 2 goes away, making it size 2:
1 2 3 4
1 2 3
1 3
Now say you want to delete indices 2 then 4. 2 goes away, so now it's size 3. Now we delete index 4.... WAIT A SECOND!
1 2 3 4
1 3 4
Index 4 doesn't exist, because we shortened the array. The element formerly known as the 4th element is now at index 3.
By deleting elements, you're changing the indexing of all elements from that point forward. Solution: save all the indices as you compute, and delete them all afterwards:
for ...
clipped_indices(end+1) = i;
end
arrayClipped(:, clipped_indices) = [];
Or, for a cleaner and faster implementation, don't make arrayClipped at all. Instead, make an logical array of all ones of the correct size, and zero them as you find the elements to clip, then index baseline2 with the resulting logical array at the end.
My working solution inspired by the answer:
%Data points on the rise and fall of the square wave are recursively clipped to the
%baseline of their respective arrays. With each loop the mean is reassessed
%and the end of the array clipped to the new mean.
%Clip baseline1
sizeStable = 0; %set flag
while sizeStable == 0
arrayMean = mean(baseline1(1,:)); %(re)calculate mean
x=size(baseline1,2); %(re)calculate size
meanCrossed = 0; %(re)set
for i=x:-1:1 %from x to 1 in steps of -1
if baseline1(1,i)>arrayMean && meanCrossed == 0 %if the data point is greater than the mean and the mean has not been crossed before
baseline1(:,i)=0; %make the entire column at that index zero
else
meanCrossed = 1;
end
end
baseline1( :, ~any(baseline1,1) ) = []; %delete columns with zeros
if size(baseline1,2) == x
sizeStable = 1;
end
end
Related
I'd like to implement a cellular automaton (CA) in Julia. Dimensions should be wrapped, this means: the left neighbor of the leftmost cell is the rightmost cell etc.
One crucial question is: how to get the neighbors of one cell to compute it's state in the next generation? As dimensions should be wrapped and Julia does not allow negative indices (as in Python) i had this idea:
Considered a 1D CA, one generation is a one-dimensional array:
0 0 1 0 0
What if we create a two dimensional Array, where the first row is shifted right and the third is shifted left, like this:
0 0 0 1 0
0 0 1 0 0
0 1 0 0 0
Now, the first column contain the states of the first cell and it's neighbors etc.
i think this can easily be generalized for two and more dimensions.
First question: do you think this is a good idea, or is this a wrong track?
EDIT: Answer to first question was no, second Question and code example discarded.
Second question: If the approach is basically ok, please have a look at the following sketch:
EDIT: Other approach, here is a stripped down version of a 1D CA, using mod1() for getting neighborhood-indices, as Bogumił Kamiński suggested.
for any cell:
- A array of all indices
- B array of all neighborhood states
- C states converted to one integer
- D lookup next state
function digits2int(digits, base=10)
int = 0
for digit in digits
int = int * base + digit
end
return int
end
gen = [0,0,0,0,0,1,0,0,0,0,0]
rule = [0,1,1,1,1,0,0,0]
function nextgen(gen, rule)
values = [mod1.(x .+ [-1,0,1], size(gen)) for x in 1:length(gen)] # A
values = [gen[value] for value in values] # B
values = [digits2int(value, 2) for value in values] # C
values = [rule[value+1] for value in values] # D
return values
end
for _ in 1:100
global gen
println(gen)
gen = nextgen(gen, rule)
end
Next step should be to extend it to two dimensions, will try it now...
The way I typically do it is to use mod1 function for wrapped indexing.
In this approach, no matter what dimensionality of your array a is then when you want to move from position x by delta dx it is enough to write mod1(x+dx, size(a, 1)) if x is the first dimension of an array.
Here is a simple example of a random walk on a 2D torus counting the number of times a given cell was visited (here I additionally use broadcasting to handle all dimensions in one expression):
function randomwalk()
a = zeros(Int, 8, 8)
pos = (1,1)
for _ in 1:10^6
# Von Neumann neighborhood
dpos = rand(((1,0), (-1,0), (0,1), (0,-1)))
pos = mod1.(pos .+ dpos, size(a))
a[pos...] += 1
end
a
end
Usually, if the CA has cells that are only dependent on the cells next to them, it's simpler just to "wrap" the vector by adding the last element to the front and the first element to the back, doing the simulation, and then "unwrap" by taking the first and last elements away again to get the result length the same as the starting array length. For the 1-D case:
const lines = 10
const start = ".........#........."
const rules = [90, 30, 14]
rule2poss(rule) = [rule & (1 << (i - 1)) != 0 for i in 1:8]
cells2bools(cells) = [cells[i] == '#' for i in 1:length(cells)]
bools2cells(bset) = prod([bset[i] ? "#" : "." for i in 1:length(bset)])
function transform(bset, ruleposs)
newbset = map(x->ruleposs[x],
[bset[i + 1] * 4 + bset[i] * 2 + bset[i - 1] + 1
for i in 2:length(bset)-1])
vcat(newbset[end], newbset, newbset[1])
end
const startset = cells2bools(start)
for rul in rules
println("\nUsing Rule $rul:")
bset = vcat(startset[end], startset, startset[1]) # wrap ends
rp = rule2poss(rul)
for _ in 1:lines
println(bools2cells(bset[2:end-1])) # unwrap ends
bset = transform(bset, rp)
end
end
As long as only the adjacent cells are used in the simulation for any given cell, this is correct.
If you extend this to a 2D matrix, you would also "wrap" the first and last rows as well as the first and last columns, and so forth.
I have a big matrix (500212x7) and a column vector like below
matrix F vector P
0001011 4
0001101 3
1101100 6
0000110 1
1110000 7
The vector contains indices considered within the matrix rows. P(1) is meant to point at F(1,4), P(2) at F(2,3) and so on.
I want to negate a bit in each row in F in a column pointed by P element (in the same row).
I thought of things like
F(:,P(1)) = ~F(:,P(1));
F(:,P(:)) = ~F(:,P(:));
but of course these scenarios won't produce the result I expect as the first line won't make P element change and the second one won't even let me start the program because a full vector cannot make an index.
The idea is I need to do this for all F and P rows (changing/incrementing "simultaneously") but take the value of P element.
I know this is easily achieved with for loop but due to large dimensions of the F array such a way to solve the problem is completely unacceptable.
Is there any kind of Matlab wizardry that lets solving such a task with the use of matrix operations?
I know this is easily achieved with for loop but due to large dimensions of the F array such a way to solve the problem is completely unacceptable.
You should never make such an assumption. First implement the loop, then check to see if it really is too slow for you or not, then worry about optimizing.
Here I'm comparing Luis' answer and the trival loop:
N = 500212;
F = rand(N,7) > 0.6;
P = randi(7,N,1);
timeit(#()method1(F,P))
timeit(#()method2(F,P))
function F = method1(F,P)
ind = (1:size(F,1)) + (P(:).'-1)*size(F,1); % create linear index
F(ind) = ~F(ind); % negate those entries
end
function F = method2(F,P)
for ii = 1:numel(P)
F(ii,P(ii)) = ~F(ii,P(ii));
end
end
Timings are 0.0065 s for Luis' answer, and 0.0023 s for the loop (MATLAB Online R2019a).
It is especially true for very large arrays, that loops are faster than vectorized code, if the vectorization requires creating an intermediate array. Memory access is expensive, using more of it makes the code slower.
Lessons: don't dismiss loops, don't prematurely try to optimize, and don't optimize without comparing.
Another solution:
xor( F, 1:7 == P )
Explanation:
1:7 == P generates one-hot arrays.
xor will cause a bit to retain its value against a 0, and flip it against a 1
Not sure if it qualifies as wizardry, but linear indexing does exactly what you want:
F = [0 0 0 1 0 1 1; 0 0 0 1 1 0 1; 1 1 0 1 1 0 0; 0 0 0 0 1 1 0; 1 1 1 0 0 0 0];
P = [4; 3; 6; 1; 7];
ind = (1:size(F,1)) + (P(:).'-1)*size(F,1); % create linear index
F(ind) = ~F(ind); % negate those entries
Is there any way to find the longest subarray of 1's in log(n) time?
example:
110011111000 - then the output is 5 (from pos 5 to 10)
1110011101 - the output here is 3 but after rotation 1 the array becomes 111100111 and the output is now 4.
001111 - the output here is 4 from pos 3 to 6 but after rotation it becomes 3 from pos 4 to 6
Note: I found the longest subarray length in O(n) before rotation. How can I improve the solution after rotation if I have the past results?
You can follow those steps:
find the longest subarray of 1 (in O(n)). During that loop find the first and last instance of 0.
Start rotating and update those parameters.
If the last index is 0 search for the previous 1 to keep updated the parameters (at complexity this will not go over O(n) total.
I will assume you know how to get max subarray index and count in O(n). In addition to that, you will need to find the second largest subarray (can be done in the same loop).
Let extract 2 more params - first and last zeros (Simple code - I can attach it if you need )
When you rotate the array there are 3 option:
Nothing change
Bigger subarray created
Current biggest subarray breaks
In the first and second cases - you only need to update the params - O(1) - you can know this is the case according your params. In the third, you will need to use the second longest subarray you find (notice that only 1 subarray can be break at a time)
For example, consider you have array: 1110011101 (as your example) and you have max = 3 and maxIndex = 5. After running the getZeroIndexs function you also know that firstZeroIndex = 3 and lastZeroIndex = 8.
How our var will look like after rotate?
max = 3
maxIndex = 6
firstZeroIndex = 4 // this will increase as long as the lastZeroIndex different then the array size
lastZeroIndex = 9 //this will go up till array.size - when it those you need to loop again till you find last
In this case, the first index move to 4 whats make him bigger then max -> max = 4 and maxIndex = 0.
Now your array is : 1111001110 so lastZeroIndex = 9 as the array size so next rotation will yield:
max = 4
maxIndex = 1
firstZeroIndex = 0
lastZeroIndex = ? // here you need to loop from the end of your array till you get 0 -> O(k) in total complexity, all the run together will be O(n)
Hope it clear, if not feel free to ask!
No, because you have to know every letter value to count 1s, which is O(n) at least.
I have a csv file from which I want to get just the values different from 0 and save them within an array x. I want to store rest of the matrix elements in an array s. This is the code I'm running in Octave:
clear all;close all;clc;
datafile = csvread('data20us.csv');
datafile = datafile(3:length(datafile));
for i=1:length(datafile)
if (datafile(i) ~= 0)
% x must stock values different from 0
x(i) = datafile(i);
else
% s must stock the rest of the values
s(i) = datafile(i);
end
end
The problem I have is that the vector x gets filled with both 0 and non 0 values (there 20 values different from 0 in the file). Is there any mistake in the condition I'm putting in the if statement?
The reason why you see zeros and non-zeros is the way you are populating the array in the loop. If you read your code, if the value at position i in your CSV file (I'm assuming this is a row or column vector) is not equal to 0, position i of the vector x gets changed. Else, position i of the vector s gets changed. Because you are not pre-allocating the vector x, what happens is that the vector x should you specify a position that is larger than the length of the array gets populated by zeros up until the desired position which you finally store the value.
Observe:
>> clear x;
>> x(4) = 3
x =
0 0 0 3
>> x(7) = 10
x =
0 0 0 3 0 0 10
As you can see, by not pre-allocating x and you specify a position other than the first to place a value there, all positions in between the previous last element and your desired position get populated with zeros. Therefore, when you get to a point in your loop where you have a value that's not equal to 0, you are seeing this zeros expansion happening.
If what I'm reading about your post is correct, you want to filter out the values in the CSV file that are not equal to 0. This can be done with simple logical indexing:
x = datafile(datafile ~= 0);
This will give you the vector x with all zero values removed. Finally, you can create the s vector which is just a vector of zeros that is as long as the total number of zeroes in the vector.
s = zeros(1, nnz(datafile == 0));
If you are somehow dead set on using a loop, consider concatenating the values instead:
clear all;close all;clc;
datafile = csvread('data20us.csv');
datafile = datafile(3:length(datafile));
% New - Make x empty
x = [];
% New - Make s empty
s = [];
for i=1:length(datafile)
if (datafile(i) ~= 0)
% x must stock values different from 0
% New - concatenate
x = [x datafile(i)];
else
% s must stock the rest of the values
% New - concatenate
s = [s datafile(i)];
end
end
I have the 137x19 cell array Location(1,4).loc and I want to find the number of times that horizontal consecutive values are present in Location(1,4).loc. I have used this code:
x=Location(1,4).loc;
y={x(:,1),x(:,2)};
for ii=1:137
cnt(ii,1)=sum(strcmp(x(:,1),y{1,1}{ii,1})&strcmp(x(:,2),y{1,2}{ii,1}));
end
y={x(:,1),x(:,2),x(:,3)};
for ii=1:137
cnt(ii,2)=sum(strcmp(x(:,1),y{1,1}{ii,1})&strcmp(x(:,2),y{1,2}{ii,1})&strcmp(x(:,3),y{1,3}{ii,1}));
end
y={x(:,1),x(:,2),x(:,3),x(:,4)};
for ii=1:137
cnt(ii,3)=sum(strcmp(x(:,1),y{1,1}{ii,1})&strcmp(x(:,2),y{1,2}{ii,1})&strcmp(x(:,3),y{1,3}{ii,1})&strcmp(x(:,4),y{1,4}{ii,1}));
end
y={x(:,1),x(:,2),x(:,3),x(:,4),x(:,5)};
for ii=1:137
cnt(ii,4)=sum(strcmp(x(:,1),y{1,1}{ii,1})&strcmp(x(:,2),y{1,2}{ii,1})&strcmp(x(:,3),y{1,3}{ii,1})&strcmp(x(:,4),y{1,4}{ii,1})&strcmp(x(:,5),y{1,5}{ii,1}));
end
... continue for all the columns. This code run and gives me the correct result but it's not automated and it's slow. Can you give me ideas to automate and speed up the code?
I think I will write an answer to this since I've not done so for a while.
First convert your cell Array to a matrix,this will ease the following steps by a lot. Then diff is the way to go
A = randi(5,[137,19]);
DiffA = diff(A')'; %// Diff creates a matrix that is 136 by 19, where each consecutive value is subtracted by its previous value.
So a 0 in DiffA would represent 2 consecutive numbers in A are equal, 2 consecutive 0s would mean 3 consecutive numbers in A are equal.
idx = DiffA==0;
cnt(:,1) = sum(idx,2);
To do 3 consecutive number counts, you could do something like:
idx2 = abs(DiffA(:,1:end-1))+abs(DiffA(:,2:end)) == 0;
cnt(:,2) = sum(idx2,2);
Or use another Diff, the abs is used to avoid negative number + positive number that also happens to give 0; otherwise only 0 + 0 will give you a 0; you can now continue this pattern by doing:
idx3 = abs(DiffA(:,1:end-2))+abs(DiffA(:,2:end-1))+abs(DiffA(:,3:end)) == 0
cnt(:,3) = sum(idx3,2);
In loop format:
absDiffA = abs(DiffA)
for ii = 1:W
absDiffA = abs(absDiffA(:,1:end-1) + absDiffA(:,1+1:end));
idx = (absDiffA == 0);
cnt(:,ii) = sum(idx,2);
end
NOTE: this method counts [0,0,0] twice when evaluating 2 consecutives, and once when evaluating 3 consecutives.