I want to create an array that has incremental random steps, I've used this simple code.
t_inici=(0:10*rand:100);
The problem is that the random number keeps unchangable between steps. Is there any simple way to change the seed of the random number within each step?
If you have a set number of points, say nPts, then you could do the following
nPts = 10; % Could use 'randi' here for random number of points
lims = [0, 10] % Start and end points
x = rand(1, nPts); % Create random numbers
% Sort and scale x to fit your limits and be ordered
x = diff(lims) * ( sort(x) - min(x) ) / diff(minmax(x)) + lims(1)
This approach always includes your end point, which a 0:dx:10 approach would not necessarily.
If you had some maximum number of points, say nPtsMax, then you could do the following
nPtsMax = 1000; % Max number of points
lims = [0,10]; % Start and end points
% Could do 10* or any other multiplier as in your example in front of 'rand'
x = lims(1) + [0 cumsum(rand(1, nPtsMax))];
x(x > lims(2)) = []; % remove values above maximum limit
This approach may be slower, but is still fairly quick and better represents the behaviour in your question.
My first approach to this would be to generate N-2 samples, where N is the desired amount of samples randomly, sort them, and add the extrema:
N=50;
endpoint=100;
initpoint=0;
randsamples=sort(rand(1, N-2)*(endpoint-initpoint)+initpoint);
t_inici=[initpoint randsamples endpoint];
However not sure how "uniformly random" this is, as you are "faking" the last 2 data, to have the extrema included. This will somehow distort pure randomness (I think). If you are not necessarily interested on including the extrema, then just remove the last line and generate N points. That will make sure that they are indeed random (or as random as MATLAB can create them).
Here is an alternative solution with "uniformly random"
[initpoint,endpoint,coef]=deal(0,100,10);
t_inici(1)=initpoint;
while(t_inici(end)<endpoint)
t_inici(end+1)=t_inici(end)+rand()*coef;
end
t_inici(end)=[];
In my point of view, it fits your attempts well with unknown steps, start from 0, but not necessarily end at 100.
From your code it seems you want a uniformly random step that varies between each two entries. This implies that the number of entries that the vector will have is unknown in advance.
A way to do that is as follows. This is similar to Hunter Jiang's answer but adds entries in batches instead of one by one, in order to reduce the number of loop iterations.
Guess a number of required entries, n. Any value will do, but a large value will result in fewer iterations and will probably be more efficient.
Initiallize result to the first value.
Generate n entries and concatenate them to the (temporary) result.
See if the current entries are already too many.
If they are, cut as needed and output (final) result. Else go back to step 3.
Code:
lower_value = 0;
upper_value = 100;
step_scale = 10;
n = 5*(upper_value-lower_value)/step_scale*2; % STEP 1. The number 5 here is arbitrary.
% It's probably more efficient to err with too many than with too few
result = lower_value; % STEP 2
done = false;
while ~done
result = [result result(end)+cumsum(step_scale*rand(1,n))]; % STEP 3. Include
% n new entries
ind_final = find(result>upper_value,1)-1; % STEP 4. Index of first entry exceeding
% upper_value, if any
if ind_final % STEP 5. If non-empty, we're done
result = result(1:ind_final-1);
done = true;
end
end
In my models, one of the most repeated tasks to be done is counting the number of each element within an array. The counting is from a closed set, so I know there are X types of elements, and all or some of them populate the array, along with zeros that represent 'empty' cells. The array is not sorted in any way, and could by quite long (about 1M elements), and this task is done thousands of times during one simulation (which is also part of hundreds of simulations). The result should be a vector r of size X, so r(k) is the amount of k in the array.
Example:
For X = 9, if I have the following input vector:
v = [0 7 8 3 0 4 4 5 3 4 4 8 3 0 6 8 5 5 0 3]
I would like to get this result:
r = [0 0 4 4 3 1 1 3 0]
Note that I don't want the count of zeros, and that elements that don't appear in the array (like 2) have a 0 in the corresponding position of the result vector (r(2) == 0).
What would be the fastest way to achieve this goal?
tl;dr: The fastest method depend on the size of the array. For array smaller than 214 method 3 below (accumarray) is faster. For arrays larger than that method 2 below (histcounts) is better.
UPDATE: I tested this also with implicit broadcasting, that was introduced in 2016b, and the results are almost equal to the bsxfun approach, with no significant difference in this method (relative to the other methods).
Let's see what are the available methods to perform this task. For the following examples we will assume X has n elements, from 1 to n, and our array of interest is M, which is a column array that can vary in size. Our result vector will be spp1, such that spp(k) is the number of ks in M. Although I write here about X, there is no explicit implementation of it in the code below, I just define n = 500 and X is implicitly 1:500.
The naive for loop
The most simple and straightforward way to cope this task is by a for loop that iterate over the elements in X and count the number of elements in M that equal to it:
function spp = loop(M,n)
spp = zeros(n,1);
for k = 1:size(spp,1);
spp(k) = sum(M==k);
end
end
This is off course not so smart, especially if only little group of elements from X is populating M, so we better look first for those that are already in M:
function spp = uloop(M,n)
u = unique(M); % finds which elements to count
spp = zeros(n,1);
for k = u(u>0).';
spp(k) = sum(M==k);
end
end
Usually, in MATLAB, it is advisable to take advantage of the built-in functions as much as possible, since most of the times they are much faster. I thought of 5 options to do so:
1. The function tabulate
The function tabulate returns a very convenient frequency table that at first sight seem to be the perfect solution for this task:
function tab = tabi(M)
tab = tabulate(M);
if tab(1)==0
tab(1,:) = [];
end
end
The only fix to be done is to remove the first row of the table if it counts the 0 element (it could be that there are no zeros in M).
2. The function histcounts
Another option that can be tweaked quite easily to our need it histcounts:
function spp = histci(M,n)
spp = histcounts(M,1:n+1);
end
here, in order to count all different elements between 1 to n separately, we define the edges to be 1:n+1, so every element in X has it's own bin. We could write also histcounts(M(M>0),'BinMethod','integers'), but I already tested it, and it takes more time (though it makes the function independent of n).
3. The function accumarray
The next option I'll bring here is the use of the function accumarray:
function spp = accumi(M)
spp = accumarray(M(M>0),1);
end
here we give the function M(M>0) as input, to skip the zeros, and use 1 as the vals input to count all unique elements.
4. The function bsxfun
We can even use binary operation #eq (i.e. ==) to look for all elements from each type:
function spp = bsxi(M,n)
spp = bsxfun(#eq,M,1:n);
spp = sum(spp,1);
end
if we keep the first input M and the second 1:n in different dimensions, so one is a column vector the other is a row vector, then the function compares each element in M with each element in 1:n, and create a length(M)-by-n logical matrix than we can sum to get the desired result.
5. The function ndgrid
Another option, similar to the bsxfun, is to explicitly create the two matrices of all possibilities using the ndgrid function:
function spp = gridi(M,n)
[Mx,nx] = ndgrid(M,1:n);
spp = sum(Mx==nx);
end
then we compare them and sum over columns, to get the final result.
Benchmarking
I have done a little test to find the fastest method from all mentioned above, I defined n = 500 for all trails. For some (especially the naive for) there is a great impact of n on the time of execution, but this is not the issue here since we want to test it for a given n.
Here are the results:
We can notice several things:
Interestingly, there is a shift in the fastest method. For arrays smaller than 214 accumarray is the fastest. For arrays larger than 214 histcounts is the fastest.
As expected the naive for loops, in both versions are the slowest, but for arrays smaller than 28 the "unique & for" option is slower. ndgrid become the slowest in arrays bigger than 211, probably because of the need to store very large matrices in memory.
There is some irregularity in the way tabulate works on arrays in size smaller than 29. This result was consistent (with some variation in the pattern) in all the trials I conducted.
(the bsxfun and ndgrid curves are truncated because it makes my computer stuck in higher values, and the trend is quite clear already)
Also, notice that the y-axis is in log10, so a decrease in unit (like for arrays in size 219, between accumarray and histcounts) means a 10-times faster operation.
I'll be glad to hear in the comments for improvements to this test, and if you have another, conceptually different method, you are most welcome to suggest it as an answer.
The code
Here are all the functions wrapped in a timing function:
function out = timing_hist(N,n)
M = randi([0 n],N,1);
func_times = {'for','unique & for','tabulate','histcounts','accumarray','bsxfun','ndgrid';
timeit(#() loop(M,n)),...
timeit(#() uloop(M,n)),...
timeit(#() tabi(M)),...
timeit(#() histci(M,n)),...
timeit(#() accumi(M)),...
timeit(#() bsxi(M,n)),...
timeit(#() gridi(M,n))};
out = cell2mat(func_times(2,:));
end
function spp = loop(M,n)
spp = zeros(n,1);
for k = 1:size(spp,1);
spp(k) = sum(M==k);
end
end
function spp = uloop(M,n)
u = unique(M);
spp = zeros(n,1);
for k = u(u>0).';
spp(k) = sum(M==k);
end
end
function tab = tabi(M)
tab = tabulate(M);
if tab(1)==0
tab(1,:) = [];
end
end
function spp = histci(M,n)
spp = histcounts(M,1:n+1);
end
function spp = accumi(M)
spp = accumarray(M(M>0),1);
end
function spp = bsxi(M,n)
spp = bsxfun(#eq,M,1:n);
spp = sum(spp,1);
end
function spp = gridi(M,n)
[Mx,nx] = ndgrid(M,1:n);
spp = sum(Mx==nx);
end
And here is the script to run this code and produce the graph:
N = 25; % it is not recommended to run this with N>19 for the `bsxfun` and `ndgrid` functions.
func_times = zeros(N,5);
for n = 1:N
func_times(n,:) = timing_hist(2^n,500);
end
% plotting:
hold on
mark = 'xo*^dsp';
for k = 1:size(func_times,2)
plot(1:size(func_times,1),log10(func_times(:,k).*1000),['-' mark(k)],...
'MarkerEdgeColor','k','LineWidth',1.5);
end
hold off
xlabel('Log_2(Array size)','FontSize',16)
ylabel('Log_{10}(Execution time) (ms)','FontSize',16)
legend({'for','unique & for','tabulate','histcounts','accumarray','bsxfun','ndgrid'},...
'Location','NorthWest','FontSize',14)
grid on
1 The reason for this weird name comes from my field, Ecology. My models are a cellular-automata, that typically simulate individual organisms in a virtual space (the M above). The individuals are of different species (hence spp) and all together form what is called "ecological community". The "state" of the community is given by the number of individuals from each species, which is the spp vector in this answer. In this models, we first define a species pool (X above) for the individuals to be drawn from, and the community state take into account all species in the species pool, not only those present in M
We know that that the input vector always contains integers, so why not use this to "squeeze" a bit more performance out of the algorithm?
I've been experimenting with some optimizations of the the two best binning methods suggested by the OP, and this is what I came up with:
The number of unique values (X in the question, or n in the example) should be explicitly converted to an (unsigned) integer type.
It's faster to compute an extra bin and then discard it, than to "only process" valid values (see the accumi_new function below).
This function takes about 30sec to run on my machine. I'm using MATLAB R2016a.
function q38941694
datestr(now)
N = 25;
func_times = zeros(N,4);
for n = 1:N
func_times(n,:) = timing_hist(2^n,500);
end
% Plotting:
figure('Position',[572 362 758 608]);
hP = plot(1:n,log10(func_times.*1000),'-o','MarkerEdgeColor','k','LineWidth',2);
xlabel('Log_2(Array size)'); ylabel('Log_{10}(Execution time) (ms)')
legend({'histcounts (double)','histcounts (uint)','accumarray (old)',...
'accumarray (new)'},'FontSize',12,'Location','NorthWest')
grid on; grid minor;
set(hP([2,4]),'Marker','s'); set(gca,'Fontsize',16);
datestr(now)
end
function out = timing_hist(N,n)
% Convert n into an appropriate integer class:
if n < intmax('uint8')
classname = 'uint8';
n = uint8(n);
elseif n < intmax('uint16')
classname = 'uint16';
n = uint16(n);
elseif n < intmax('uint32')
classname = 'uint32';
n = uint32(n);
else % n < intmax('uint64')
classname = 'uint64';
n = uint64(n);
end
% Generate an input:
M = randi([0 n],N,1,classname);
% Time different options:
warning off 'MATLAB:timeit:HighOverhead'
func_times = {'histcounts (double)','histcounts (uint)','accumarray (old)',...
'accumarray (new)';
timeit(#() histci(double(M),double(n))),...
timeit(#() histci(M,n)),...
timeit(#() accumi(M)),...
timeit(#() accumi_new(M))
};
out = cell2mat(func_times(2,:));
end
function spp = histci(M,n)
spp = histcounts(M,1:n+1);
end
function spp = accumi(M)
spp = accumarray(M(M>0),1);
end
function spp = accumi_new(M)
spp = accumarray(M+1,1);
spp = spp(2:end);
end
I want to compare the pixel values of two images, which I have stored in arrays.
Suppose the arrays are A and B. I want to compare the elements one by one, and if A[l] == B[k], then I want to store the match as a key value-pair in a third array, C, like so: C[l] = k.
Since the arrays are naturally quite large, the solution needs to finish within a reasonable amount of time (minutes) on a Core 2 Duo system.
This seems to work in under a second for 1024*720 matrices:
A = randi(255,737280,1);
B = randi(255,737280,1);
C = zeros(size(A));
[b_vals, b_inds] = unique(B,'first');
for l = 1:numel(b_vals)
C(A == b_vals(l)) = b_inds(l);
end
First we find the unique values of B and the indices of the first occurrences of these values.
[b_vals, b_inds] = unique(B,'first');
We know that there can be no more than 256 unique values in a uint8 array, so we've reduced our loop from 1024*720 iterations to just 256 iterations.
We also know that for each occurrence of a particular value, say 209, in A, those locations in C will all have the same value: the location of the first occurrence of 209 in B, so we can set all of them at once. First we get locations of all of the occurrences of b_vals(l) in A:
A == b_vals(l)
then use that mask as a logical index into C.
C(A == b_vals(l))
All of these values will be equal to the corresponding index in B:
C(A == b_vals(l)) = b_inds(l);
Here is the updated code to consider all of the indices of a value in B (or at least as many as are necessary). If there are more occurrences of a value in A than in B, the indices wrap.
A = randi(255,737280,1);
B = randi(255,737280,1);
C = zeros(size(A));
b_vals = unique(B);
for l = 1:numel(b_vals)
b_inds = find(B==b_vals(l)); %// find the indices of each unique value in B
a_inds = find(A==b_vals(l)); %// find the indices of each unique value in A
%// in case the length of a_inds is greater than the length of b_inds
%// duplicate b_inds until it is larger (or equal)
b_inds = repmat(b_inds,[ceil(numel(a_inds)/numel(b_inds)),1]);
%// truncate b_inds to be the same length as a_inds (if necessary) and
%// put b_inds into the proper places in C
C(a_inds) = b_inds(1:numel(a_inds));
end
I haven't fully tested this code, but from my small samples it seems to work properly and on the full-size case, it only takes about twice as long as the previous code, or less than 2 seconds on my machine.
So, if I understand your question correctly, you want for each value of l=1:length(A) the (first) index k into B so that A(l) == B(k). Then:
C = arrayfun(#(val) find(B==val, 1, 'first'), A)
could give you your solution, as long as you're sure that every element will have a match. The above solution would fail otherwise, complaning that the function returned a non-scalar (because find would return [] if no match is found). You have two options:
Using a cell array to store the result instead of a numeric array. You would need to call arrayfun with 'UniformOutput', false at the end. Then, the values of A without matches in B would be those for which isempty(C{i}) is true.
Providing a default value for an index into A with no matches in B (e.g. 0 or NaN). I'm not sure about this one, but I think that you would need to add 'ErrorHandler', #(~,~) NaN to the arrayfun call. The error handler is a function that gets called when the function passed to arrayfun fails, and may either rethrow the error or compute a substitute value. Thus the #(~,~) NaN. I am not sure that it would work, however, since in this case the error is in arrayfun and not in the passed function, but you can try it.
If you have the images in arrays A & B
idx = A == B;
C = zeros(size(A));
C(idx) = A(idx);
For my project, I wish to quickly generate random permutations of a binary array of fixed length and a given number of 1s and 0s. Given these random permutations, I wish to add them elementwise.
I am currently using numpy's ndarray object, which is convenient for adding elementwise. My current code is as follows:
# n is the length of the array. I want to run this across a range of
# n=100 to n=1000.
row = np.zeros(n)
# m_list is a given list of integers. I am iterating over many possible
# combinations of possible values for m in m_list. For example, m_list
# could equal [5, 100, 201], for n = 500.
for m in m_list:
row += np.random.permutation(np.concatenate([np.ones(m), np.zeros(n - m)]))
My question is, is there any faster way to do this? According to timeit, 1000000 calls of "np.random.permutation(np.concatenate([np.ones(m), np.zeros(n - m)]))" takes 49.6 seconds. For my program's purposes, I'd like to decrease this by an order of magnitude. Can anyone suggest a faster way to do this?
Thank you!
For me version with array allocation outside the loop
was faster but not much - 8% or so, using cProfile
row = np.zeros(n, dtype=np.float64)
wrk = np.zeros(n, dtype=np.float64)
for m in m_list:
wrk[0:m] = 1.0
wrk[m:n] = 0.0
row += np.random.permutation(wrk)
You might try to shuffle(wrk) in-place instead of returning another array from permutation, but for me difference was negligible
I'm writing a function that requires some values in a matrix of arbitrary dimansions to be dropped in a specified dimension.
For example, say I have a 3x3 matrix:
a=[1,2,3;4,5,6;7,8,9];
I might want to drop the third element in each row, in which case I could do
a = a(:,1:2)
But what if the dimensions of a are arbitrary, and the dimension to trim is defined as an argument in the function?
Using linear indexing, and some carefully considered maths is an option but I was wondering if there is a neater soltion?
For those interested, this is my current code:
...
% Find length in each dimension
sz = size(dat);
% Get the proportion to trim in each dimension
k = sz(d)*abs(p);
% Get the decimal part and integer parts of k
int_part = fix(k);
dec_part = abs(k - int_part);
% Sort the array
dat = sort(dat,d);
% Trim the array in dimension d
if (int_part ~=0)
switch d
case 1
dat = dat(int_part + 1 : sz(1) - int_part,:);
case 2
dat = dat(:,int_part + 1 : sz(2) - int_part);
end
end
...
It doesn't get any neater than this:
function A = trim(A, n, d)
%// Remove n-th slice of A in dimension d
%// n can be vector of indices. d needs to be scalar
sub = repmat({':'}, 1, ndims(A));
sub{d} = n;
A(sub{:}) = [];
This makes use of the not very well known fact that the string ':' can be used as an index. With due credit to this answer by #AndrewJanke, and to #chappjc for bringing it to my attention.
a = a(:, 1:end-1)
end, used as a matrix index, always refers to the index of the last element of that matrix
if you want to trim different dimensions, the simplest way is using and if/else block - as MatLab only supports 7 dimensions at most, you wont need an infinite number of these to cover all bases
The permute function allows to permute the dimension of an array of any dimension.
You can place the dimension you want to trim in a prescribed position (the first, I guess), trim, and finally restore the original ordering. In this way you can avoid running loops and do what you want compactly.