Conversion from 3D cell array to a set of 2D matrices - arrays

I have a 3D-cell array designated as A{s,i,h}, serving as a store for large amounts of numerical data during a nested-loop portion of my script. Some of the cell entries will be blank [ ], whilst the rest consist of numbers - either singular or in arrays (1 x 10 double etc.):
I want to convert this cell array to a set of 2D matrices.
Specifically, one separate matrix for each value of h (h is always equal 1:3) and one column in each matrix for every value of s. Each column will contain all the numerical data combined - it does not need to be separated by i.
How can I go about this? I ordinarily deal with 3D-cell arrays in this form to produce separate matrices (one for each value of h) using something like this:
lens = sum(cellfun('length',reshape(A,[],size(A,3))),1);
max_length = max(lens);
mat = zeros(max_length,numel(lens));
mask = bsxfun(#le,[1:max_length]',lens);
mat(mask) = [A{:}];
mat(mat==0) = NaN;
mat = sort(mat*100);
Matrix1 = mat(~isnan(mat(:,1)),1);
Matrix2 = mat(~isnan(mat(:,2)),2);
Matrix3 = mat(~isnan(mat(:,3)),3);
However in this instance, each matrix had only a single column. I'm have trouble adding multiple columns to each output matrix.

1. Result in the form of a cell array of matrices (as requested)
Here's one possible approach. I had to use one for loop. However, the loop can be easily avoided if you accept a 3D-array result instead of a cell array of 2D-arrays. See second part of the answer.
If you follow the comments in the code and inspect the result of each step, it's straightforward to see how it works.
%// Example data
A(:,:,1) = { 1:2, 3:5, 6:9; 10 11:12 13:15 };
A(:,:,2) = { 16:18, 19:22, 23; 24:28, [], 29:30 };
%// Let's go
[S, I, H] = size(A);
B = permute(A, [2 1 3]); %// permute rows and columns
B = squeeze(mat2cell(B, I, ones(1, S), ones(1, H))); %// group each col of B into a cell...
B = cellfun(#(x) [x{:}], B, 'uniformoutput', false); %// ...containing a single vector
t = cellfun(#numel, B); %// lengths of all columns of result
result = cell(1,H); %// preallocate
for h = 1:H
mask = bsxfun(#le, (1:max(t(:,h))), t(:,h)).'; %'// values of result{h} to be used
result{h} = NaN(size(mask)); %// unused values will be NaN
result{h}(mask) = [B{:,h}]; %// fill values for matrix result{h}
end
Result in this example:
A{1,1,1} =
1 2
A{2,1,1} =
10
A{1,2,1} =
3 4 5
A{2,2,1} =
11 12
A{1,3,1} =
6 7 8 9
A{2,3,1} =
13 14 15
A{1,1,2} =
16 17 18
A{2,1,2} =
24 25 26 27 28
A{1,2,2} =
19 20 21 22
A{2,2,2} =
[]
A{1,3,2} =
23
A{2,3,2} =
29 30
result{1} =
1 10
2 11
3 12
4 13
5 14
6 15
7 NaN
8 NaN
9 NaN
result{2} =
16 24
17 25
18 26
19 27
20 28
21 29
22 30
23 NaN
2. Result in the form of 3D array
As indicated above, using a 3D array to store the result permits avoiding loops. In the code below, the last three lines replace the loop used in the first part of the answer. The rest of the code is the same.
%// Example data
A(:,:,1) = { 1:2, 3:5, 6:9; 10 11:12 13:15 };
A(:,:,2) = { 16:18, 19:22, 23; 24:28, [], 29:30 };
%// Let's go
[S, I, H] = size(A);
B = permute(A, [2 1 3]); %// permute rows and columns
B = squeeze(mat2cell(B, I, ones(1, S), ones(1, H))); %// group each col of B into a cell...
B = cellfun(#(x) [x{:}], B, 'uniformoutput', false); %// ...containing a single vector
t = cellfun(#numel, B); %// lengths of all columns of result
mask = bsxfun(#le, (1:max(t(:))).', permute(t, [3 1 2])); %'// values of result to be used
result = NaN(size(mask)); %// unused values will be NaN
result(mask) = [B{:}]; %// fill values
This gives (compare with result of the first part):
>> result
result(:,:,1) =
1 10
2 11
3 12
4 13
5 14
6 15
7 NaN
8 NaN
9 NaN
result(:,:,2) =
16 24
17 25
18 26
19 27
20 28
21 29
22 30
23 NaN
NaN NaN

Brute force approach:
[num_s, num_i, num_h] = size(A);
cellofmat = cell(num_h,1);
for matrix = 1:num_h
sizemat = max(cellfun(#numel, A(:,1,matrix)));
cellofmat{matrix} = nan(sizemat, num_s);
for column = 1:num_s
lengthcol = length(A{column, 1, matrix});
cellofmat{matrix}(1:lengthcol, column) = A{column, 1,matrix};
end
end
Matrix1 = cellofmat{1};
Matrix2 = cellofmat{2};
Matrix3 = cellofmat{3};
I don't know what your actual structure looks like but this works for A that is setup using the following steps.
A = cell(20,1,3);
for x = 1:3
for y = 1:20
len = ceil(rand(1,1) * 10);
A{y,1,x} = rand(len, 1);
end
end

Related

MATLAB: extract values from 3d matrix at given row and column indcies using sub2ind 3d

I have 3d matrix A that has my data. At multiple locations defined by row and column indcies as shown by matrix row_col_idx I want to extract all data along the third dimension as shown below:
A = cat(3,[1:3;4:6], [7:9;10:12],[13:15;16:18],[19:21;22:24]) %matrix(2,3,4)
row_col_idx=[1 1;1 2; 2 3];
idx = sub2ind(size(A(:,:,1)), row_col_idx(:,1),row_col_idx(:,2));
out=nan(size(A,3),size(row_col_idx,1));
for k=1:size(A,3)
temp=A(:,:,k);
out(k,:)=temp(idx);
end
out
The output of this code is as follows:
A(:,:,1) =
1 2 3
4 5 6
A(:,:,2) =
7 8 9
10 11 12
A(:,:,3) =
13 14 15
16 17 18
A(:,:,4) =
19 20 21
22 23 24
out =
1 2 6
7 8 12
13 14 18
19 20 24
The output is as expected. However, the actual A and row_col_idx are huge, so this code is computationally expensive. Is there away to vertorize this code to avoid the loop and the temp matrix?
This can be vectorized using linear indexing and implicit expansion:
out = A( row_col_idx(:,1) + ...
(row_col_idx(:,2)-1)*size(A,1) + ...
(0:size(A,1)*size(A,2):numel(A)-1) ).';
The above builds an indexing matrix as large as the output. If this is unacceptable due to memory limiations, it can be avoided by reshaping A:
sz = size(A); % store size A
A = reshape(A, [], sz(3)); % collapse first two dimensions
out = A(row_col_idx(:,1) + (row_col_idx(:,2)-1)*sz(1),:).'; % linear indexing along
% first two dims of A
A = reshape(A, sz); % reshape back A, if needed
A more efficient method is using the entries of the row_col_idx vector for selecting the elements from A. I have compared the two methods for a large matrix, and as you can see the calculation is much faster.
For the A given in the question, it gives the same output
A = rand([2,3,10000000]);
row_col_idx=[1 1;1 2; 2 3];
idx = sub2ind(size(A(:,:,1)), row_col_idx(:,1),row_col_idx(:,2));
out=nan(size(A,3),size(row_col_idx,1));
tic;
for k=1:size(A,3)
temp=A(:,:,k);
out(k,:)=temp(idx);
end
time1 = toc;
%% More efficient method:
out2 = nan(size(A,3),size(row_col_idx,1));
tic;
for jj = 1:size(row_col_idx,1)
out2(:,jj) = [A(row_col_idx(jj,1),row_col_idx(jj,2),:)];
end
time2 = toc;
fprintf('Time calculation 1: %d\n',time1);
fprintf('Time calculation 2: %d\n',time2);
Gives as output:
Time calculation 1: 1.954714e+01
Time calculation 2: 2.998120e-01

alternating and shifting sections of an array

I have a n x m array (could be any size array but it will not be a 1 x m) and I want to rotate / shift each square loop individually no matter the array size.
How can I alternate the rotation / shift each square loop no matter the size of the array.
Please note: I'm not trying to calculate the values in the array but shift the values.
My thought process was to get the values of each "square loop" and place them into one row and do a circshift then place them back into another array.
I ran into problems trying to get the values back into the original n x m array size and I wasn't sure how I could loop through the process for different n x m arrays.
The pink highlighted section, left of the arrows is the starting position of the array and it's "loops" and the green highlighted section, right of the arrows is the type of rotation / shift of the values that I'm trying to create. The array could have more than 3 "loops" this is just an example.
Code below:
I=[1:5;6:10;11:15;16:20;21:25;26:30]
[rw,col] = size(I);
outer_1=[I(1,:),I(2:end-1,end).',I(end,end:-1:1),I(end-1:-1:2,1).'] %get values in one row (so I can shift values)
outer_1_shift=circshift(outer_1,[0 1]) %shift values
new_array=zeros(rw,col);
Ps: I'm using Octave 4.2.2 Ubuntu 18.04
Edit: The circshift function was changed for Octave 5.0, the last edit made it compatible with previous versions
1;
function r = rndtrip (n, m, v)
rv = #(x) x - 2 * (v - 1);
r = [v * ones(1,rv(m)-1) v:n-v+1 (n-v+1)*ones(1,rv(m)-2)];
if (rv(m) > 1)
r = [r n-v+1:-1:v+1];
endif
endfunction
function idx = ring (n, m , v)
if (2*(v-1) > min (n, m))
r = [];
else
r = rndtrip (n, m, v);
c = circshift (rndtrip (m, n, v)(:), - n + 2 * v - 1).';
idx = sub2ind ([n m], r, c);
endif
endfunction
# your I
I = reshape (1:30, 5, 6).';
# positive is clockwise, negative ccw
r = [1 -1 1];
for k = 1:numel(r)
idx = ring (rows(I), columns(I), k);
I(idx) = I(circshift(idx(:), r(k)));
endfor
I
gives
I =
6 1 2 3 4
11 8 9 14 5
16 7 18 19 10
21 12 13 24 15
26 17 22 23 20
27 28 29 30 25
run it on tio
So, I had the same idea as in Andy's comment. Nevertheless, since I was already preparing some code, here is my suggestion:
% Input.
I = reshape(1:30, 5, 6).'
[m, n] = size(I);
% Determine number of loops.
nLoops = min(ceil([m, n] / 2));
% Iterate loops.
for iLoop = 1:nLoops
% Determine number of repetitions per row / column.
row = n - 2 * (iLoop - 1);
col = m - 2 * (iLoop - 1);
% Initialize indices.
idx = [];
% Add top row indices.
idx = [idx, [repelem(iLoop, row).']; iLoop:(n-(iLoop-1))];
% Add right column indices.
idx = [idx, [[iLoop+1:(m-(iLoop-1))]; repelem(n-(iLoop-1), col-1).']];
if (iLoop != m-(iLoop-1))
% Add bottom row indices.
idx = [idx, [repelem(m-(iLoop-1), row-1).'; (n-(iLoop-1)-1:-1:iLoop)]]
end
if (iLoop != n-(iLoop-1))
% Add left column indices.
idx = [idx, [[(m-(iLoop-1))-1:-1:iLoop+1]; repelem(iLoop, col-2).']]
end
% Convert subscript indices to linear indices.
idx = sub2ind(size(I), idx(1, :), idx(2, :));
% Determine direction for circular shift operation.
if (mod(iLoop, 2) == 1)
direction = [0 1];
else
direction = [0 -1];
end
% Replace values in I.
I(idx) = circshift(I(idx), direction);
end
% Output.
I
Unfortunately, I couldn't think of a smarter way to generate the indices, since you need to maintain the right order and avoid double indices. As you can see, I obtain subscript indices with respect to I, since this can be done quite easy using the matrix dimensions and number of loops. Nevertheless, for the circshift operation and later replacing of the values in I, linear indices are more handy, so that's why the sub2ind operation.
Input and output look like this:
I =
1 2 3 4 5
6 7 8 9 10
11 12 13 14 15
16 17 18 19 20
21 22 23 24 25
26 27 28 29 30
I =
6 1 2 3 4
11 8 9 14 5
16 7 18 19 10
21 12 13 24 15
26 17 22 23 20
27 28 29 30 25
I was right, that the "shift direction" changes with every loop?
Hope that helps!
Caution: I haven't tested for generality, yet. So, please report any errors you might come across.

Reshape vector for given duration values in Matlab

I have the following vector:
x = [6 7 8 9 10 11 17 18 19 20];
I have the duration vector, d, which counts the elements in x:
d = [6 4] => d = [x(1:6) x(7:10)];
So I want to find the entrances and exits of d(1) and d(2):
d(1) = x(1:6) => r1 = [6 11];
d(2) = x(7:10) => r2 = [17 20];
So finally I want to reshape into the following matrix:
result = [r1; r2] = [6 11; 17 20];
Anyone has any idea?
One way of constructing your result would be to use cumsum to help construct your indices.
result = x([cumsum([1 d(1:end-1)]); cumsum(d)]).';
6 11
17 20
If you don't mind breaking this into two lines, you could easily do the same with the following which may be more performant since you're only computing the cumulative sum once.
C = cumsum(d);
result = x([1, C(1:end-1)+1; C]).';
This constructs a matrix containing the first and last indices of each group specified in d. We can then use these to index directly into x.
starts = cumsum([1 d(1:end-1)])
1 7
ends = cumsum(d)
6 10
indices = [starts; ends]
1 7
6 10
x(indices).'
6 11
17 20

MATLAB Merging Arrays

I am unable to figure out how to merge two arrays. My data is like this with arrays A and B.
A = [ 0 0; 0 0; 2 2; 2 2;]
B = [ 1 1; 1 1; 3 3; 3 3; 4 4; 4 4; 5 5; 5 5;]
and I need the final array "C" to look like this after merging:
C = [ 0 0; 0 0; 1 1; 1 1; 2 2; 2 2; 3 3; 3 3; 4 4; 4 4; 5 5; 5 5;]
I've tried using different ways with reshaping each array and trying to use a double loop but haven't got it to work yet.
In my actual data it is inserting 9 rows of array B following 3 rows of array A and then repeated 100 times. So, there are 12 new merged rows (3 rows from array A and 9 rows from array B) repeated 100 times with a final row number == 1200. Array A actual data has 300 rows and actual Array B data has 900 rows
thanks,
Here's a solution using only reshape:
A = [ 6 6; 3 3; 5 5; 4 4;]
B = [ 0 0; 21 21; 17 17; 33 33; 29 29; 82 82;]
A_count = 2;
B_count = 3;
w = size(A,2); %// width = number of columns
Ar = reshape(A,A_count,w,[]);
Br = reshape(B,B_count,w,[]);
Cr = [Ar;Br];
C = reshape(Cr,[],w)
The [] in reshape means "how ever many you need to get to the total number of elements". So if we have 12 elements in B and do:
Br = reshape(B,3,2,[]);
We're reshaping B into a 3x2xP 3-dimensional matrix. Since the total number of elements is 12, P = 2 because 12 = 3x2x2.
Output:
A =
6 6
3 3
5 5
4 4
B =
0 0
21 21
17 17
33 33
29 29
82 82
C =
6 6
3 3
0 0
21 21
17 17
5 5
4 4
33 33
29 29
82 82
Approach #1
This could be one approach assuming I got the requirements of the problem right -
%// Inputs
A = [ 6 6; 3 3; 5 5; 4 4;];
B = [ 0 0; 21 21; 17 17; 33 33; 29 29; 82 82;];
%// Parameters that decide at what intervals to "cut" A and B along the rows
A_cutlen = 2; %// Edit this to 3 for the actual data
B_cutlen = 3; %// Edit this to 9 for the actual data
%// Cut A and B along the rows at specified intervals into 3D arrays
A3d = permute(reshape(A,A_cutlen,size(A,1)/A_cutlen,[]),[1 3 2])
B3d = permute(reshape(B,B_cutlen,size(B,1)/B_cutlen,[]),[1 3 2])
%// Vertically concatenate those 3D arrays to get a 3D array
%// version of expected output, C
C3d = [A3d;B3d]
%// Convert the 3D array to a 2D array which is the final output
C_out = reshape(permute(C3d,[1 3 2]),size(C3d,1)*size(C3d,3),[])
Sample run -
A =
6 6
3 3
5 5
4 4
B =
0 0
21 21
17 17
33 33
29 29
82 82
A_cutlen =
2
B_cutlen =
3
C_out =
6 6
3 3
0 0
21 21
17 17
5 5
4 4
33 33
29 29
82 82
Approach #2
Just for the love of bsxfun, here's one approach with it and ones (no reshape or permute) -
%// Assuming A_cutlen and B_cutlen decide cutting intervals for A and B
%// Concatenate A and B along rows
AB = [A;B]
%// Find the row indices corresponding to rows from A and B to be placed
%// according to the problem requirements
idx1 = [1:A_cutlen size(A,1)+[1:B_cutlen]]
idx2 = [A_cutlen*ones(1,A_cutlen) B_cutlen*ones(1,B_cutlen)]
idx = bsxfun(#plus,idx1(:),idx2(:)*[0:size(A,1)/A_cutlen-1])
%// Re-arrange AB based on "idx" for the desired output
C = AB(idx,:)
based on your new criteria this is what you want. My solution isn't the nicest looking (maye someone can think of a nice vectorized approach), but it works
a_step = 2;
b_step = 3;
C = zeros(size([A;B]));
%we use two iterators, one for each matrix, they must be initialized to 1
a_idx = 1;
b_idx = 1;
%this goes through the entire c array doing a_step+b_step rows at a
%time
for c_idx=1:a_step+b_step :size(C,1)-1
%this takes the specified number of rows from a
a_part = A(a_idx:a_idx+a_step-1,:);
%tkaes the specified number of rows from b
b_part = B(b_idx:b_idx+b_step-1,:);
%combines the parts together in the appropriate spot in c
C(c_idx:c_idx + a_step + b_step -1,:) = [a_part;b_part];
%advances the "iterator" on the a and b matricies
a_idx = a_idx + a_step;
b_idx = b_idx + b_step;
end
using
A = [ 6 6; 3 3; 5 5; 4 4;]
B = [ 0 0; 21 21; 17 17; 33 33; 29 29; 82 82;]
produces
C =[6 6; 3 3; 0 0; 21 21; 17 17; 5 5; 4 4; 33 33; 29 29; 82 82;]

finding index-positions of a composed-matrix

I need help with my code. The code is used to find the minumin of a square-distance problem. I am providing my code through an example, I believe this will be the easiest way to explain what I need.
clear all
clc
x=10.8; % is a fixed value
y=34; % is a fixed value
z=12; % is a fixed value
A = [11 14 1; 5 8 18; 10 8 19; 13 20 16]; % a (4x3) matrix
B = [2 3 10; 6 15 16; 7 3 15; 14 14 19]; % a (4x3) matrix
I create a new matrix C which is composed in this following way:
C1 = bsxfun(#minus, A(:,1)',B(:,1));
C1=C1(:); % this is the first column of the new matrix C
C2 = bsxfun(#minus, A(:,2)',B(:,2));
C2=C2(:); % this is the second column of the new matrix C
C3 = bsxfun(#minus, A(:,3)',B(:,3));
C3=C3(:); % this is the third column of the new matrix C
C = [C1 C2 C3]; % the new matrix C of size (16x3)
C has to be formed in this way! And this is what I meant when I wrote in my title a composed-matrix
Then:
[d,p] = min((C(:,1)-x).^2 + (C(:,2)-y).^2 + (C(:,3)-z).^2);
d = sqrt(d);
outputs:
d = 18.0289;
p = 13;
Gives me the distance (d) and position (p) which satisfies this min problem.
MY PROBLEM:
I need to find which combinations of A and B has given my this p value, in other words I need the index from ´A,B´ which gives me this optimal C1,C2,C3:
C1 = bsxfun(#minus, A(?,1)',B(?,1));
C2 = bsxfun(#minus, A(?,2)',B(?,2));
C3 = bsxfun(#minus, A(?,3)',B(?,3));
The ? is the index position I need, in this case the index position of the matrix A and the index position of B.
Calculated by hand I have the following illustration:
I know that:
C = [9 11 -9
5 -1 -15
4 11 -14
-3 0 -18
3 5 8
-1 -7 2
-2 5 3
-9 -6 -1
8 5 9
4 -7 3
3 5 4
-4 -6 0
11 17 6
7 5 0
6 17 1
-1 6 -3]
And I know that my optimal index is given in the position 13th. This index positions goes back to:
[13-2 20-3 16-10]
Which is A(4,:) - B(1,:)
I need a code which can help me to find this indexes from A and B
Thanks in advance!
PS. I am using the code in parameter estimation problems of ODEs.
First case: vector-matrix case
subvals = bsxfun(#minus,A,[x y z])
[distance,index] = min(sqrt(sum(subvals.^2,2)))
Second case: Two matrices case
subvals = bsxfun(#minus,A,permute(B,[3 2 1]));
[distances,indices] = min(sqrt(sum(subvals.^2,2)),[],3)
Testing for second case:
%%// Get some random data into A and B
A = randi(20,8,3)
B = randi(20,4,3)
%%// Just to test out out code for correctness,
%%// let us make any one one row of B, say 3rd row equal to
%%// any one row of A, say the 6th row -
B(3,:) = A(6,:)
%%// Use the earlier code
subvals = bsxfun(#minus,A,permute(B,[3 2 1]));
[distances,indices] = min(sqrt(sum(subvals.^2,2)),[],3)
%%// Get the minimum row index for A and B
[~,min_rowA] = min(distances)
min_rowB = indices(min_rowA)
Verification
min_rowA =
6
min_rowB =
3
Edit 1 [Response to simple example posted in question]:
The title says you are interested in finding the difference of two matrices and then find the shortest distance between it to a vector [x y z]. So I am hoping this is what you need -
x=10.8;
y=34;
z=12;
A = [11 14 1; 5 8 18; 10 8 19; 13 20 16];
B = [2 3 10; 6 15 16; 7 3 15; 14 14 19];
C = A -B; %%// Distance of two vectors as posted in title
subvals = bsxfun(#minus,C,[x y z])
[distance,index] = min(sqrt(sum(subvals.^2,2)))
Output
distance =
31.0780
index =
3
Edit 2: After you have done this -
[d,p] = min((C(:,1)-x).^2 + (C(:,2)-y).^2 + (C(:,3)-z).^2);
If you are looking to find the corresponding indices of A and B , you may do this -
[minindex_alongB,minindex_alongA] = ind2sub(size(A),p)

Resources