MATLAB: extract values from 3d matrix at given row and column indcies using sub2ind 3d - arrays

I have 3d matrix A that has my data. At multiple locations defined by row and column indcies as shown by matrix row_col_idx I want to extract all data along the third dimension as shown below:
A = cat(3,[1:3;4:6], [7:9;10:12],[13:15;16:18],[19:21;22:24]) %matrix(2,3,4)
row_col_idx=[1 1;1 2; 2 3];
idx = sub2ind(size(A(:,:,1)), row_col_idx(:,1),row_col_idx(:,2));
out=nan(size(A,3),size(row_col_idx,1));
for k=1:size(A,3)
temp=A(:,:,k);
out(k,:)=temp(idx);
end
out
The output of this code is as follows:
A(:,:,1) =
1 2 3
4 5 6
A(:,:,2) =
7 8 9
10 11 12
A(:,:,3) =
13 14 15
16 17 18
A(:,:,4) =
19 20 21
22 23 24
out =
1 2 6
7 8 12
13 14 18
19 20 24
The output is as expected. However, the actual A and row_col_idx are huge, so this code is computationally expensive. Is there away to vertorize this code to avoid the loop and the temp matrix?

This can be vectorized using linear indexing and implicit expansion:
out = A( row_col_idx(:,1) + ...
(row_col_idx(:,2)-1)*size(A,1) + ...
(0:size(A,1)*size(A,2):numel(A)-1) ).';
The above builds an indexing matrix as large as the output. If this is unacceptable due to memory limiations, it can be avoided by reshaping A:
sz = size(A); % store size A
A = reshape(A, [], sz(3)); % collapse first two dimensions
out = A(row_col_idx(:,1) + (row_col_idx(:,2)-1)*sz(1),:).'; % linear indexing along
% first two dims of A
A = reshape(A, sz); % reshape back A, if needed

A more efficient method is using the entries of the row_col_idx vector for selecting the elements from A. I have compared the two methods for a large matrix, and as you can see the calculation is much faster.
For the A given in the question, it gives the same output
A = rand([2,3,10000000]);
row_col_idx=[1 1;1 2; 2 3];
idx = sub2ind(size(A(:,:,1)), row_col_idx(:,1),row_col_idx(:,2));
out=nan(size(A,3),size(row_col_idx,1));
tic;
for k=1:size(A,3)
temp=A(:,:,k);
out(k,:)=temp(idx);
end
time1 = toc;
%% More efficient method:
out2 = nan(size(A,3),size(row_col_idx,1));
tic;
for jj = 1:size(row_col_idx,1)
out2(:,jj) = [A(row_col_idx(jj,1),row_col_idx(jj,2),:)];
end
time2 = toc;
fprintf('Time calculation 1: %d\n',time1);
fprintf('Time calculation 2: %d\n',time2);
Gives as output:
Time calculation 1: 1.954714e+01
Time calculation 2: 2.998120e-01

Related

alternating and shifting sections of an array

I have a n x m array (could be any size array but it will not be a 1 x m) and I want to rotate / shift each square loop individually no matter the array size.
How can I alternate the rotation / shift each square loop no matter the size of the array.
Please note: I'm not trying to calculate the values in the array but shift the values.
My thought process was to get the values of each "square loop" and place them into one row and do a circshift then place them back into another array.
I ran into problems trying to get the values back into the original n x m array size and I wasn't sure how I could loop through the process for different n x m arrays.
The pink highlighted section, left of the arrows is the starting position of the array and it's "loops" and the green highlighted section, right of the arrows is the type of rotation / shift of the values that I'm trying to create. The array could have more than 3 "loops" this is just an example.
Code below:
I=[1:5;6:10;11:15;16:20;21:25;26:30]
[rw,col] = size(I);
outer_1=[I(1,:),I(2:end-1,end).',I(end,end:-1:1),I(end-1:-1:2,1).'] %get values in one row (so I can shift values)
outer_1_shift=circshift(outer_1,[0 1]) %shift values
new_array=zeros(rw,col);
Ps: I'm using Octave 4.2.2 Ubuntu 18.04
Edit: The circshift function was changed for Octave 5.0, the last edit made it compatible with previous versions
1;
function r = rndtrip (n, m, v)
rv = #(x) x - 2 * (v - 1);
r = [v * ones(1,rv(m)-1) v:n-v+1 (n-v+1)*ones(1,rv(m)-2)];
if (rv(m) > 1)
r = [r n-v+1:-1:v+1];
endif
endfunction
function idx = ring (n, m , v)
if (2*(v-1) > min (n, m))
r = [];
else
r = rndtrip (n, m, v);
c = circshift (rndtrip (m, n, v)(:), - n + 2 * v - 1).';
idx = sub2ind ([n m], r, c);
endif
endfunction
# your I
I = reshape (1:30, 5, 6).';
# positive is clockwise, negative ccw
r = [1 -1 1];
for k = 1:numel(r)
idx = ring (rows(I), columns(I), k);
I(idx) = I(circshift(idx(:), r(k)));
endfor
I
gives
I =
6 1 2 3 4
11 8 9 14 5
16 7 18 19 10
21 12 13 24 15
26 17 22 23 20
27 28 29 30 25
run it on tio
So, I had the same idea as in Andy's comment. Nevertheless, since I was already preparing some code, here is my suggestion:
% Input.
I = reshape(1:30, 5, 6).'
[m, n] = size(I);
% Determine number of loops.
nLoops = min(ceil([m, n] / 2));
% Iterate loops.
for iLoop = 1:nLoops
% Determine number of repetitions per row / column.
row = n - 2 * (iLoop - 1);
col = m - 2 * (iLoop - 1);
% Initialize indices.
idx = [];
% Add top row indices.
idx = [idx, [repelem(iLoop, row).']; iLoop:(n-(iLoop-1))];
% Add right column indices.
idx = [idx, [[iLoop+1:(m-(iLoop-1))]; repelem(n-(iLoop-1), col-1).']];
if (iLoop != m-(iLoop-1))
% Add bottom row indices.
idx = [idx, [repelem(m-(iLoop-1), row-1).'; (n-(iLoop-1)-1:-1:iLoop)]]
end
if (iLoop != n-(iLoop-1))
% Add left column indices.
idx = [idx, [[(m-(iLoop-1))-1:-1:iLoop+1]; repelem(iLoop, col-2).']]
end
% Convert subscript indices to linear indices.
idx = sub2ind(size(I), idx(1, :), idx(2, :));
% Determine direction for circular shift operation.
if (mod(iLoop, 2) == 1)
direction = [0 1];
else
direction = [0 -1];
end
% Replace values in I.
I(idx) = circshift(I(idx), direction);
end
% Output.
I
Unfortunately, I couldn't think of a smarter way to generate the indices, since you need to maintain the right order and avoid double indices. As you can see, I obtain subscript indices with respect to I, since this can be done quite easy using the matrix dimensions and number of loops. Nevertheless, for the circshift operation and later replacing of the values in I, linear indices are more handy, so that's why the sub2ind operation.
Input and output look like this:
I =
1 2 3 4 5
6 7 8 9 10
11 12 13 14 15
16 17 18 19 20
21 22 23 24 25
26 27 28 29 30
I =
6 1 2 3 4
11 8 9 14 5
16 7 18 19 10
21 12 13 24 15
26 17 22 23 20
27 28 29 30 25
I was right, that the "shift direction" changes with every loop?
Hope that helps!
Caution: I haven't tested for generality, yet. So, please report any errors you might come across.

reversing shuffling of array by indexing

I have a matrix whose columns which was shuffled according to some index. I know want to find the index that 'unshuffles' the array back into its original state.
For example:
myArray = [10 20 30 40 50 60]';
myShuffledArray = nan(6,3)
myShufflingIndex = nan(6,3)
for x = 1:3
myShufflingIndex(:,x) = randperm(length(myArray))';
myShuffledArray(:,x) = myArray(myShufflingIndex(:,x));
end
Now I want to find a matrix myUnshufflingIndex, which reverses the shuffling to get an array myUnshuffledArray = [10 20 30 40 50 60; 10 20 30 40 50 60; 10 20 30 40 50 60]'
I expect to use myUnshufflingIndex in the following way:
for x = 1:3
myUnShuffledArray(:,x) = myShuffledArray(myUnshufflingIndex(:,x), x);
end
For example, if one column in myShufflingIndex = [2 4 6 3 5 1]', then the corresponding column in myUnshufflingIndex is [6 1 4 2 5 3]'
Any ideas on how to get myUnshufflingIndex in a neat vectorised way? Also, is there a better way to unshuffle the array columnwise than in a loop?
You can get myUnshufflingIndex with a single call to sort:
[~, myUnshufflingIndex] = sort(myShufflingIndex, 1);
Alternatively, you don't even need to compute myUnshufflingIndex, since you can just use myShufflingIndex on the left hand side of the assignment to unshuffle the data:
for x = 1:3
myUnShuffledArray(myShufflingIndex(:, x), x) = myShuffledArray(:, x);
end
And if you'd like to avoid a for loop while unshuffling, you can vectorize it by adding an offset to each column of your index, turning it into a matrix of linear indices instead of just row indices:
[nRows, nCols] = size(myShufflingIndex);
myUnshufflingIndex = myShufflingIndex+repmat(0:nRows:(nRows*(nCols-1)), nRows, 1);
myUnShuffledArray = nan(nRows, nCols); % Preallocate
myUnShuffledArray(myUnshufflingIndex) = myShuffledArray;

what is the matlab way without loop to multiply a matrix with a vector (extending to the 3rd dim)?

A simple example to illustrate all elements of a matrix multiplying each element of a vector to generate a 3D array.
M = reshape(1:12,4,3);
V = 1:2;
n = length(V);
A = nan([size(M),n]);
for ii = 1 : n
A(:,:,ii) = M * V(ii);
end
then
A(:,:,1) =
1 5 9
2 6 10
3 7 11
4 8 12
A(:,:,2) =
2 10 18
4 12 20
6 14 22
8 16 24
Or by repmat both M and V to the size of [4,3,2],
A = repmat(M,1,1,n) * reshape(V(ones(size(M)),:),[size(M),n])
It creates two 3D array by repmat besides the resulting 3d array A.
How to make it efficiently WITHOUT for loop and save the memory use?
According to the answer by #Lincoln,
A = bsxfun(#times, repmat(M,1,1,n), reshape(1:n, 1, 1, n));
repmat the vector V to 3d is not necessary.
Is it possible to create NO 3d array if the final result wanted is 2d, the sum of A along the 3rd dim? By for loop, the code would be
M = reshape(1:12,4,3);
V = 1:2;
n = length(V);
A = 0;
for ii = 1 : n
A = A + M * V(ii);
end
Try replacing the for.. loop with:
M_rep = repmat(M,1,1,n) %//repeat M in the 3rd dimension n times
v = reshape(1:n, 1, 1, n) %//create a vector [1 2 .. n] pointing in the third dimension
A = bsxfun(#times, M_rep, v) %//vector multiply them in the third dimension
In your above example, n=2.
EDIT (to your added question): To sum without allocating A:
B = sum(bsxfun(#times, M_rep, v),3);

Conversion from 3D cell array to a set of 2D matrices

I have a 3D-cell array designated as A{s,i,h}, serving as a store for large amounts of numerical data during a nested-loop portion of my script. Some of the cell entries will be blank [ ], whilst the rest consist of numbers - either singular or in arrays (1 x 10 double etc.):
I want to convert this cell array to a set of 2D matrices.
Specifically, one separate matrix for each value of h (h is always equal 1:3) and one column in each matrix for every value of s. Each column will contain all the numerical data combined - it does not need to be separated by i.
How can I go about this? I ordinarily deal with 3D-cell arrays in this form to produce separate matrices (one for each value of h) using something like this:
lens = sum(cellfun('length',reshape(A,[],size(A,3))),1);
max_length = max(lens);
mat = zeros(max_length,numel(lens));
mask = bsxfun(#le,[1:max_length]',lens);
mat(mask) = [A{:}];
mat(mat==0) = NaN;
mat = sort(mat*100);
Matrix1 = mat(~isnan(mat(:,1)),1);
Matrix2 = mat(~isnan(mat(:,2)),2);
Matrix3 = mat(~isnan(mat(:,3)),3);
However in this instance, each matrix had only a single column. I'm have trouble adding multiple columns to each output matrix.
1. Result in the form of a cell array of matrices (as requested)
Here's one possible approach. I had to use one for loop. However, the loop can be easily avoided if you accept a 3D-array result instead of a cell array of 2D-arrays. See second part of the answer.
If you follow the comments in the code and inspect the result of each step, it's straightforward to see how it works.
%// Example data
A(:,:,1) = { 1:2, 3:5, 6:9; 10 11:12 13:15 };
A(:,:,2) = { 16:18, 19:22, 23; 24:28, [], 29:30 };
%// Let's go
[S, I, H] = size(A);
B = permute(A, [2 1 3]); %// permute rows and columns
B = squeeze(mat2cell(B, I, ones(1, S), ones(1, H))); %// group each col of B into a cell...
B = cellfun(#(x) [x{:}], B, 'uniformoutput', false); %// ...containing a single vector
t = cellfun(#numel, B); %// lengths of all columns of result
result = cell(1,H); %// preallocate
for h = 1:H
mask = bsxfun(#le, (1:max(t(:,h))), t(:,h)).'; %'// values of result{h} to be used
result{h} = NaN(size(mask)); %// unused values will be NaN
result{h}(mask) = [B{:,h}]; %// fill values for matrix result{h}
end
Result in this example:
A{1,1,1} =
1 2
A{2,1,1} =
10
A{1,2,1} =
3 4 5
A{2,2,1} =
11 12
A{1,3,1} =
6 7 8 9
A{2,3,1} =
13 14 15
A{1,1,2} =
16 17 18
A{2,1,2} =
24 25 26 27 28
A{1,2,2} =
19 20 21 22
A{2,2,2} =
[]
A{1,3,2} =
23
A{2,3,2} =
29 30
result{1} =
1 10
2 11
3 12
4 13
5 14
6 15
7 NaN
8 NaN
9 NaN
result{2} =
16 24
17 25
18 26
19 27
20 28
21 29
22 30
23 NaN
2. Result in the form of 3D array
As indicated above, using a 3D array to store the result permits avoiding loops. In the code below, the last three lines replace the loop used in the first part of the answer. The rest of the code is the same.
%// Example data
A(:,:,1) = { 1:2, 3:5, 6:9; 10 11:12 13:15 };
A(:,:,2) = { 16:18, 19:22, 23; 24:28, [], 29:30 };
%// Let's go
[S, I, H] = size(A);
B = permute(A, [2 1 3]); %// permute rows and columns
B = squeeze(mat2cell(B, I, ones(1, S), ones(1, H))); %// group each col of B into a cell...
B = cellfun(#(x) [x{:}], B, 'uniformoutput', false); %// ...containing a single vector
t = cellfun(#numel, B); %// lengths of all columns of result
mask = bsxfun(#le, (1:max(t(:))).', permute(t, [3 1 2])); %'// values of result to be used
result = NaN(size(mask)); %// unused values will be NaN
result(mask) = [B{:}]; %// fill values
This gives (compare with result of the first part):
>> result
result(:,:,1) =
1 10
2 11
3 12
4 13
5 14
6 15
7 NaN
8 NaN
9 NaN
result(:,:,2) =
16 24
17 25
18 26
19 27
20 28
21 29
22 30
23 NaN
NaN NaN
Brute force approach:
[num_s, num_i, num_h] = size(A);
cellofmat = cell(num_h,1);
for matrix = 1:num_h
sizemat = max(cellfun(#numel, A(:,1,matrix)));
cellofmat{matrix} = nan(sizemat, num_s);
for column = 1:num_s
lengthcol = length(A{column, 1, matrix});
cellofmat{matrix}(1:lengthcol, column) = A{column, 1,matrix};
end
end
Matrix1 = cellofmat{1};
Matrix2 = cellofmat{2};
Matrix3 = cellofmat{3};
I don't know what your actual structure looks like but this works for A that is setup using the following steps.
A = cell(20,1,3);
for x = 1:3
for y = 1:20
len = ceil(rand(1,1) * 10);
A{y,1,x} = rand(len, 1);
end
end

Partial sum of divisions of a vector

If there is a vector like this,
T = [1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16]
(the size of vector T can be flexible)
How can I get a array of 'sum of divisions'?
For example,
fn(T, 5) = [ (1+2+3+4+5) , (6+7+8+9+10), (11+12+13+14+15) , 16]
One option, which doesn't require the padding of zeros on the original array, is the use of accumarray and ceil:
div = 5;
out = accumarray(ceil((1:numel(T))/div).',T(:))
Another option using cumsum and diff instead:
div = 5;
T(ceil(numel(T)/div)*div) = 0;
cs = cumsum(T)
out = diff( [0 cs(div:div:end) ] )
Edit: once the padding is done, cumsum and diff are a little overkill and one should proceed as in Bentoy's answer.
Another way, close to the 2nd option of thewaywewalk:
div = 5;
T(ceil(numel(T)/div)*div) = 0;
out = sum(reshape(T,div,[])).'; % transpose if you really want a column vector
Also, one one-liner solution (I prefer this one):
out = blockproc(T,[1 5], #(blk) sum(blk.data), 'PadPartialBlocks',true);
Don't forget to set the parameter 'PadPartialBlocks', this is the key of avoiding explicit padding.
There is an in-built function vec2mat in Communications System Toolbox to convert a vector into a 2D matrix that cuts off after every N elements and puts into separate rows, padding the leftover places at the end with zeros to maintain 2D size . So, after using vec2mat, summing all the rows would be enough to give you the desired output. Here's the implementation -
sum(vec2mat(T,5),2)
Sample run -
>> T = 1:16;
>> vec2mat(T,5)
ans =
1 2 3 4 5
6 7 8 9 10
11 12 13 14 15
16 0 0 0 0
>> sum(vec2mat(T,5),2)
ans =
15
40
65
16

Resources