Say i have a N-Dimensional matrix A that can be of any size. For example:
A = rand([2,5,3]);
I want to calculate all possible pairwise differences between elements of the matrix, along a given dimension. For example, if i wanted to calculate the differences along dimension 3, a shortcut would be to create a matrix like so:
B = cat(3, A(:,:,2) - A(:,:,1), A(:,:,3) - A(:,:,1), A(:,:,3) - A(:,:,2));
However, i want this to be able to operate along any dimension, with a matrix of any size. So, ideally, i'd like to either create a function that takes in a matrix A and calculates all pairwise differences along dimension DIM, or find a builtin MATLAB function that does the same thing.
The diff function seems like it could be useful, but it only calculates differences between adjacent elements, not all possible differences.
Doing my research on this issue, i have found a couple of posts about getting all possible differences, but most of these are for items in a vector (and ignore the dimensionality issue). Does anyone know of a quick fix?
Specific Dimension Cases
If you don't care about a general solution, for a dim=3 case, it would be as simple as couple lines of code -
dim = 3
idx = fliplr(nchoosek(1:size(A,dim),2))
B = A(:,:,idx(:,1)) - A(:,:,idx(:,2))
You can move around those idx(..) to specific dimension positions, if you happen to know the dimension before-hand. So, let's say dim = 4, then just do -
B = A(:,:,:,idx(:,1)) - A(:,:,:,idx(:,2))
Or let's say dim = 3, but A is a 4D array, then do -
B = A(:,:,idx(:,1),:) - A(:,:,idx(:,2),:)
Generic Case
For a Nth dim case, it seems you need to welcome a party of reshapes and permutes -
function out = pairwise_diff(A,dim)
%// New permuting dimensions
new_permute = [dim setdiff(1:ndims(A),dim)];
%// Permuted A and its 2D reshaped version
A_perm = permute(A,new_permute);
A_perm_2d = reshape(A_perm,size(A,dim),[]);
%// Get pairiwse indices for that dimension
N = size(A,dim);
[Y,X] = find(bsxfun(#gt,[1:N]',[1:N])); %//' OR fliplr(nchoosek(1:size(A,dim),2))
%// Get size of new permuted array that would have the length of
%// first dimension equal to number of such pairwise combinations
sz_A_perm = size(A_perm);
sz_A_perm(1) = numel(Y);
%// Get the paiwise differences; reshape to a multidimensiona array of same
%// number of dimensions as the input array
diff_mat = reshape(A_perm_2d(Y,:) - A_perm_2d(X,:),sz_A_perm);
%// Permute back to original dimension sequence as the final output
[~,return_permute] = sort(new_permute);
out = permute(diff_mat,return_permute);
return
So much for a generalization , huh!
Related
Say A is a 3x4x5 array. I am given a vector a, say of dimension 2 and b of dimension 2. If I do A(a,b,:) it will give 5 matrices of dimensions 2x2. I instead want the piecewise vectors (without writing a for loop).
So, I want the two vectors of A which are given by (a's first element and b's first element) and (a's second element and b's second element)
How do I do this without a for loop? If A were two dimensions I could do this using sub2ind. I don't know how to access the entire vectors.
You can use sub2ind to find the linear index to the first element of each output vector: ind = sub2ind(size(A),a,b). To get the whole vectors, you can't do A(ind,:), because the : has to be the 3rd dimension. However, what you can do is reshape A to be 2D, collapsing the first two dimensions into one. We have a linear index to the vectors we want, that will correctly index the first dimension of this reshaped A:
% input:
A = rand(3,4,5);
a = [2,3];
b = [1,2];
% expected:
B = [squeeze(A(a(1),b(1),:)).';squeeze(A(a(2),b(2),:)).']
% solution:
ind = sub2ind(size(A),a,b);
C = reshape(A,[],size(A,3));
C = C(ind,:)
assert(isequal(B,C))
You can change a and b to be 3d arrays just like A and then the sub2ind should be able to index the whole matrix. Like this:
Edit: Someone pointed out a bug. I have changed it so that a correction gets added. The problem was that ind1, which should have had the index number for each desired element of A was only indexing the first "plane" of A. The fix is that for each additional "plane" in the z direction, the total number of elements in A in the previous "planes" must be added to the index.
A=rand(3,4,5);
a=[2,3];
b=[1,2];
a=repmat(a,1,1,size(A,3));
b=repmat(b,1,1,size(A,3));
ind1=sub2ind(size(A),a,b);
correction=(size(A,1)*size(A,2))*(0:size(A,3)-1);
correction=permute(correction,[3 1 2]);
ind1=ind1+repmat(correction,1,2,1);
out=A(ind1)
I have a two arrays within a <1x2 cell>. I want to permute those arrays. Of course, I could use a loop to permute each one, but is there any way to do that task at once, without using loops?
Example:
>> whos('M')
Name Size Bytes Class Attributes
M 1x2 9624 cell
>> permute(M,p_matrix)
This does not permute the contents of the two arrays within M.
I could use something like:
>> for k=1:size(M,2), M{k} = permute(M{k},p_matrix); end
but I'd prefer not to use loops.
Thanks.
This seems to work -
num_cells = numel(M) %// Number of cells in input cell array
size_cell = size(M{1}) %// Get sizes
%// Get size of the numeric array that will hold all of the data from the
%// input cell array with the second dimension representing the index of
%// each cell from the input cell array
size_num_arr = [size_cell(1) num_cells size_cell(2:end)]
%// Dimensions array for permuting with the numeric array holding all data
perm_dim = [1 3:numel(size_cell)+1 2]
%// Store data from input M into a vertically concatenated numeric array
num_array = vertcat(M{:})
%// Reshape and permute the numeric array such that the index to be used
%// for indexing data from different cells ends up as the final dimension
num_array = permute(reshape(num_array,size_num_arr),perm_dim)
num_array = permute(num_array,[p_matrix numel(size_cell)+1])
%// Save the numeric array as a cell array with each block from
%// thus obtained numeric array from its first to the second last dimension
%// forming each cell
size_num_arr2 = size(num_array)
size_num_arr2c = num2cell(size_num_arr2(1:end-1))
M = squeeze(mat2cell(num_array,size_num_arr2c{:},ones(1,num_cells)))
Some quick tests show that mat2cell would prove to be the bottleneck, so if you don't mind indexing into the intermediate numeric array variable num_array and use it's last dimension for an equivalent indexing into M, then this approach could be useful.
Now, another approach if you would like to preserve the cell format would be with arrayfun, assuming each cell of M to be a 4D numeric array -
M = arrayfun(#(x) num_array(:,:,:,:,x),1:N,'Uniform',0)
This seems to perform much better than with mat2cell in terms of performance.
Please note that arrayfun isn't a vectorized solution as most certainly it uses loops behind-the-scenes and seems like mat2cell is using for loops inside its source code, so please do keep all these issues in mind.
In MATLAB, I am using the shake.m function (http://www.mathworks.com/matlabcentral/fileexchange/10067-shake) to randomly shuffle each column. For example:
a = [1 2 3; 4 5 6; 7 8 9]
a =
1 2 3
4 5 6
7 8 9
b = shake(a)
b =
7 8 6
1 5 9
4 2 3
This function does exactly what I want, however my columns are very long (>10,000,000) and so this takes a long time to run. Does anyone know of a faster way of achieving this? I have tried shaking each column vector separately but this isn't faster. Thanks!
You can use randperm like this, but I don't know if it will be any faster than shake:
[m,n]=size(a)
for c = 1:n
a(randperm(m),c) = a(:,c);
end
Or you can try switch the randperm around to see which is faster (should produce the same result):
[m,n]=size(a)
for c = 1:n
a(:,c) = a(randperm(m),c);
end
Otherwise how many rows do you have? If you have far fewer rows than columns, it's possible that we can assume each permutation will be repeated, so what about something like this:
[m,n]=size(a)
cols = randperm(n);
k = 5; %//This is a parameter you'll need to tweak...
set_size = floor(n/k);
for set = 1:set_size:n
set_cols = cols(set:(set+set_size-1))
a(:,set_cols) = a(randperm(m), set_cols);
end
which would massively reduce the number of calls to randperm. Breaking it up into k equal sized sets might not be optimal though, you might want to add some randomness to that as well. The basic idea here though is that there will only be factorial(m) different orderings, and if m is much smaller than n (e.g. m=5, n=100000 like your data), then these orderings will be repeated naturally. So instead of letting that occur by itself, rather manage the process and reduce the calls to randperm which would be producing the same result anyway.
Here's a simple vectorized approach. Note that it creates an auxiliary matrix (ind) the same size as a, so depending on your memory it may be usable or not.
[~, ind] = sort(rand(size(a))); %// create a random sorting for each column
b = a(bsxfun(#plus, ind, 0:size(a,1):numel(a)-1)); %// convert to linear index
Obtain shuffled indices using randperm
idx = randperm(size(a,1));
Use the indices to shuffle the vector:
m = size(a,1);
for i=1:m
b(:,i) = a(randperm(m,:);
end
Look at this answer: Matlab: How to random shuffle columns of matrix
Here's a no-loop approach as it processes all indices at once and I believe this is as random as one could get given the requirements of shuffling among each column only.
Code
%// Get sizes
[m,n] = size(a);
%// Create an array of randomly placed sequential indices from 1 to numel(a)
rand_idx = randperm(m*n);
%// segregate those indices into rows and cols for the size of input data, a
col = ceil(rand_idx/m);
row = rem(rand_idx,m);
row(row==0)=m;
%// Sort both these row and col indices based on col, such that we have col
%// as 1,1,1,1 ...2,2,2,....3,3,3,3 and so on, which would represent per col
%// indices for the input data. Use these indices to linearly index into a
[scol,ind1] = sort(col);
a(1:m*n) = a((scol-1)*m + row(ind1))
Final output is obtained in a itself.
I'm writing a function that requires some values in a matrix of arbitrary dimansions to be dropped in a specified dimension.
For example, say I have a 3x3 matrix:
a=[1,2,3;4,5,6;7,8,9];
I might want to drop the third element in each row, in which case I could do
a = a(:,1:2)
But what if the dimensions of a are arbitrary, and the dimension to trim is defined as an argument in the function?
Using linear indexing, and some carefully considered maths is an option but I was wondering if there is a neater soltion?
For those interested, this is my current code:
...
% Find length in each dimension
sz = size(dat);
% Get the proportion to trim in each dimension
k = sz(d)*abs(p);
% Get the decimal part and integer parts of k
int_part = fix(k);
dec_part = abs(k - int_part);
% Sort the array
dat = sort(dat,d);
% Trim the array in dimension d
if (int_part ~=0)
switch d
case 1
dat = dat(int_part + 1 : sz(1) - int_part,:);
case 2
dat = dat(:,int_part + 1 : sz(2) - int_part);
end
end
...
It doesn't get any neater than this:
function A = trim(A, n, d)
%// Remove n-th slice of A in dimension d
%// n can be vector of indices. d needs to be scalar
sub = repmat({':'}, 1, ndims(A));
sub{d} = n;
A(sub{:}) = [];
This makes use of the not very well known fact that the string ':' can be used as an index. With due credit to this answer by #AndrewJanke, and to #chappjc for bringing it to my attention.
a = a(:, 1:end-1)
end, used as a matrix index, always refers to the index of the last element of that matrix
if you want to trim different dimensions, the simplest way is using and if/else block - as MatLab only supports 7 dimensions at most, you wont need an infinite number of these to cover all bases
The permute function allows to permute the dimension of an array of any dimension.
You can place the dimension you want to trim in a prescribed position (the first, I guess), trim, and finally restore the original ordering. In this way you can avoid running loops and do what you want compactly.
How can I squeeze only a subset of singleton dimensions of a matrix in Matlab? The squeeze function removes them all.
I keep the index to those dimensions in a vector called "dims".
Code
%// Input matrix is assumed as A
sz = size(A)
t2 = sz~=1
t2(dims)=1
out = reshape(A,sz(t2)) %// out is the desired output
If you are crazy about dense codes, you can try this -
sz = size(A)
out = reshape(A,sz(sort([dims find(sz~=1)])))
In Matlab, there is no tailing singleton dimension. A n*m*1 matrix is automatically a n*m matrix. Knowing this, your problem could be solved permuting the dimensions you don't want to the end:
X=ones(2,1,2,1,2,1,2,1,2,1)
%dimensions you want to keep in any case
dims=[2:4];
%Notice, S is [2,1,2,1,2,1,2,1,2], last dimension already "gone"
S=size(X)
%keep if size>1
dimensions_to_keep=S>1
%and keep if in "dims" list
dimensions_to_keep(dims)=1
%now permute dimensions you don't want to the end
Y=permute(X,[find(dimensions_to_keep),find(~dimensions_to_keep)])