mean of parts of an array in octave - arrays

Question

I have two arrays. One is a list of lengths within the other. For example
zarray = [1 2 3 4 5 6 7 8 9 10]
and
lengths = [1 3 2 1 3]
I want to average (mean) over parts the first array with lengths given by the second. For this example, resulting in:
[mean([1]),mean([2,3,4]),mean([5,6]),mean([7]),mean([8,9,10])]
I am trying to avoid looping, for the sake of speed. I tried using mat2cell and cellfun as follows
zcell = mat2cell(zarray,[1],lengths);
zcellsum = cellfun('mean',zcell);
But the cellfun part is very slow. Is there a way to do this without looping or cellfun?

Answer 1

Here is a fully vectorized solution (no explicit for-loops, or hidden loops with ARRAYFUN, CELLFUN, ..). The idea is to use the extremely fast ACCUMARRAY function:
%# data
zarray = [1 2 3 4 5 6 7 8 9 10];
lengths = [1 3 2 1 3];
%# generate subscripts: 1 2 2 2 3 3 4 5 5 5
endLocs = cumsum(lengths(:));
subs = zeros(endLocs(end),1);
subs([1;endLocs(1:end-1)+1]) = 1;
subs = cumsum(subs);
%# mean of each part
means = accumarray(subs, zarray) ./ lengths(:)
The result in this case:
means =
1
3
5.5
7
9
Speed test:
Consider the following comparison of the different methods. I am using the TIMEIT function by Steve Eddins:
function [t,v] = testMeans()
%# generate test data
[arr,len] = genData();
%# define functions
f1 = #() func1(arr,len);
f2 = #() func2(arr,len);
f3 = #() func3(arr,len);
f4 = #() func4(arr,len);
%# timeit
t(1) = timeit( f1 );
t(2) = timeit( f2 );
t(3) = timeit( f3 );
t(4) = timeit( f4 );
%# return results to check their validity
v{1} = f1();
v{2} = f2();
v{3} = f3();
v{4} = f4();
end
function [arr,len] = genData()
%#arr = [1 2 3 4 5 6 7 8 9 10];
%#len = [1 3 2 1 3];
numArr = 10000; %# number of elements in array
numParts = 500; %# number of parts/regions
arr = rand(1,numArr);
len = zeros(1,numParts);
len(1:end-1) = diff(sort( randperm(numArr,numParts) ));
len(end) = numArr - sum(len);
end
function m = func1(arr, len)
%# #Drodbar: for-loop
idx = 1;
N = length(len);
m = zeros(1,N);
for i=1:N
m(i) = mean( arr(idx+(0:len(i)-1)) );
idx = idx + len(i);
end
end
function m = func2(arr, len)
%# #user1073959: MAT2CELL+CELLFUN
m = cellfun(#mean, mat2cell(arr, 1, len));
end
function m = func3(arr, len)
%# #Drodbar: ARRAYFUN+CELLFUN
idx = arrayfun(#(a,b) a-(0:b-1), cumsum(len), len, 'UniformOutput',false);
m = cellfun(#(a) mean(arr(a)), idx);
end
function m = func4(arr, len)
%# #Amro: ACCUMARRAY
endLocs = cumsum(len(:));
subs = zeros(endLocs(end),1);
subs([1;endLocs(1:end-1)+1]) = 1;
subs = cumsum(subs);
m = accumarray(subs, arr) ./ len(:);
if isrow(len)
m = m';
end
end
Below are the timings. Tests were performed on a WinXP 32-bit machine with MATLAB R2012a. My method is an order of magnitude faster than all other methods. For-loop is second best.
>> [t,v] = testMeans();
>> t
t =
0.013098 0.013074 0.022407 0.00031807
| | | \_________ #Amro: ACCUMARRAY (!)
| | \___________________ #Drodbar: ARRAYFUN+CELLFUN
| \______________________________ #user1073959: MAT2CELL+CELLFUN
\__________________________________________ #Drodbar: FOR-loop
Furthermore all results are correct and equal -- differences are in the order of eps the machine precision (caused by different ways of accumulating round-off errors), therefore considered rubbish and simply ignored:
%#assert( isequal(v{:}) )
>> maxErr = max(max( diff(vertcat(v{:})) ))
maxErr =
3.3307e-16

Answer 2

Here is a solution using arrayfun and cellfun
zarray = [1 2 3 4 5 6 7 8 9 10];
lengths = [1 3 2 1 3];
% Generate the indexes for the elements contained within each length specified
% subset. idx would be {[1], [4, 3, 2], [6, 5], [7], [10, 9, 8]} in this case
idx = arrayfun(#(a,b) a-(0:b-1), cumsum(lengths), lengths,'UniformOutput',false);
means = cellfun( #(a) mean(zarray(a)), idx);
Your desired output result:
means =
1.0000 3.0000 5.5000 7.0000 9.0000
Following #tmpearce comment I did a quick time performance comparison between above's solution, from which I create a function called subsetMeans1
function means = subsetMeans1( zarray, lengths)
% Generate the indexes for the elements contained within each length specified
% subset. idx would be {[1], [4, 3, 2], [6, 5], [7], [10, 9, 8]} in this case
idx = arrayfun(#(a,b) a-(0:b-1), cumsum(lengths), lengths,'UniformOutput',false);
means = cellfun( #(a) mean(zarray(a)), idx);
and a simple for loop alternative, function subsetMeans2.
function means = subsetMeans2( zarray, lengths)
% Method based on single loop
idx = 1;
N = length(lengths);
means = zeros( 1, N);
for i = 1:N
means(i) = mean( zarray(idx+(0:lengths(i)-1)) );
idx = idx+lengths(i);
end
Using the next test scrip, based on TIMEIT, that allows checking performance varying the number of elements on the input vector and sizes of elements per subset:
% Generate some data for the performance test
% Total of elements on the vector to test
nVec = 100000;
% Max of elements per subset
nSubset = 5;
% Data generation aux variables
lenghtsGen = randi( nSubset, 1, nVec);
accumLen = cumsum(lenghtsGen);
maxIdx = find( accumLen < nVec, 1, 'last' );
% % Original test data
% zarray = [1 2 3 4 5 6 7 8 9 10];
% lengths = [1 3 2 1 3];
% Vector to test
zarray = 1:nVec;
lengths = [ lenghtsGen(1:maxIdx) nVec-accumLen(maxIdx)] ;
% Double check that nVec is will be the max index
assert ( sum(lengths) == nVec)
t1(1) = timeit(#() subsetMeans1( zarray, lengths));
t1(2) = timeit(#() subsetMeans2( zarray, lengths));
fprintf('Time spent subsetMeans1: %f\n',t1(1));
fprintf('Time spent subsetMeans2: %f\n',t1(2));
It turns out that the non-vectorised version without arrayfun and cellfun is faster, presumably due to the extra overhead of those functions
Time spent subsetMeans1: 2.082457
Time spent subsetMeans2: 1.278473