This is the code:
A = rand(3,3,3);
P(1) = max(max(A(:,:,1)));
P(2) = max(max(A(:,:,2)));
P(3) = max(max(A(:,:,3)));
You can create P in one call:
%fastest solution for size(A,1)>size(A,2)
P = max(max(A,[],1),[],2)
%fastest solution for size(A,2)>size(A,1)
P = max(max(A,[],2),[],1)
For large matrices, it is faster to have a small intermediate result (the output of the first max call)
One approach is to collapse first two dimensions into one, and maximize along that dimension. I haven't tested it for speed, though.
P = max(reshape(A,[],size(A,3)));
Related
I have a sparse matrix in MATLAB:
N=1000;
P=0.01;
A=sprand(N,N,P);
and I want to change all non zero entries at certain columns into ones.
That is, something like this:
c=randi(N,[1,round(N/10)]);
A(non zeros at columns c)=1;
Of course it can be done in a for loop, but that's clearly not the solution I'm looking for.
I tried several solutions using nnz, nonzeros, spfun - but with no soccess.
Can anyone come up with a simple way to do it?
Thanks,
Elad
You can do it this way:
A(:,c) = abs(sign(A(:,c))); % take the absolute value of the sign for all entries
% in the submatrix defined by the columns in c, and
% then assign the result back
Equivalently,
A(:,c) = logical(A(:,c);
or
A(:,c) = A(:,c)~=0;
These may not be fast, because they process all entries in those columns, not just the nonzero entries. Dohyun's approach is probably faster.
You can try this
N = 1000;
P = 0.01;
A = sprand(N,N,P);
c = unique(randi(N,[1,round(N/10)]))'; % sorted column index
[r,cind] = find(A(:,c));
A(sub2ind([N,N],r,c(cind)))=1;
Related to Luis Mendos answer, but a bit simpler
A(:,c) = ceil(A(:,c));
I have an array that I iteratively build up like follows:
step1.shape = (200,200)
step2.shape = (200,200,200)
step3.shape = (200,200,200,200)
and then reshape to:
step4.shape = (200,200**3)
I do this because dask.array.atop doesn't seem to allow you to go from a shape like this: (200,200) -> (200,200**2). I think this is so that it is related to chunking and lazy evaluation.
When I do step4 and try to reshape it, dask seems to want to compute the matrix prior to reshaping it which results in significant computation time and memory use.
Is there a way to avoid this?
As requested, here is some dummy code:
def prod_mat(matrix_a,matrix_b):
#mat_a.shape = (300,...,300,200)
#mat_b.shape = (300, 200)
mat_a = matrix_a.reshape(-1,matrix_a.shape[-1])
#mat_a = (300**n,200)
mat_b = matrix_b.reshape(-1,matrix_b.shape[-1])
#mat_b = (300,200)
mat_temp = np.repeat(mat_a,matrix_b.shape[0],axis=0)*np.tile(mat_b.T,mat_a.shape[0]).T
new_dim = int(math.log(mat_temp.shape[0])/math.log(matrix_a.shape[0]))
new_shape = [matrix_a.shape[0] for n in range(new_dim)]
new_shape.append(-1)
result = mat_temp.reshape(tuple(new_shape))
#result.shape = (300,...,300,300,200)
return result
b = np.random.rand(300,200)
b = da.from_array(b,chunks=100)
c=da.atop(prod_mat,'ijk',b,'ik',b,'jk')
d=da.atop(prod_mat,'ijkl',c,'ijl',b,'kl')
e=da.atop(prod_mat,'ijklm',d,'ijkm',b,'lm')
f = e.sum(axis=-1)
f.reshape(300,300**3) ----> This is slow, as if it is using compute()
This computation isn't calling compute, instead it's stuck making a very very large graph. Generally speaking reshaping parallel arrays is pretty intense. Lots of your little chunks end up talking to lots of your other little chunks, creating havoc. This example is particularly bad.
Perhaps there is another way to produce your output in the correct shape initially?
Looking through the development logs it appears that this failure was actually anticipated during development: https://github.com/dask/dask/pull/758
I would like to efficiently concatenate multiple matrices into a vector. However, the number of such matrices and their sizes vary. Say, I have two stacks A and B, each consisting of m matrices.
Naive approach would be the following:
merged = [];
for i = 1 : m
merged = [merged ; A{i}(:) ; B{i}(:)];
end
The challenging part is to optimise the above code to avoid copying the older array contents to the new array as it makes each assignment. For instance, one could compute the number of elements in each matrix and then preallocate a vector capable of storing all the elements. Still, I am not entirely sure how to efficiently place the matrices inside the vector.
Any suggestions would be appreciated.
One possible approach:
merged = cellfun(#(x) x(:), [A(:) B(:)].', 'uni', false);
merged = vertcat(merged{:});
Depending on the size of A and B the follow could be faster
C = {A{:} ; B{:}};
merged = vertcat(C{:});
(starting to show above numel(A) > 500)
I used the following as test data
m = 1000;
A = cell(m,1);
B = cell(m,1);
for i=1:m
A{i} = round(10*rand(max(round(11*rand),1),1));
B{i} = round(10*rand(max(round(5*rand),1),1));
end
I would like to safe a certain amount of grayscale-images (->2D-arrays) as layers in a 3D-array.
Because it should be very fast for a realtime-application I would like to vectorize the following code, where m is the number of shifts:
for i=1:m
array(:,:,i)=imabsdiff(circshift(img1,[0 i-1]), img2);
end
nispio showed me a very advanced version, which you can see here:
I = speye(size(img1,2)); E = -1*I;
ii = toeplitz(1:m,[1,size(img1,2):-1:2]);
D = vertcat(repmat(I,1,m),E(:,ii));
data_c = shape(abs([double(img1),double(img2)]*D),size(data_r,1),size(data_r,2),m);
At the moment the results of both operations are not the same, maybe it shifts the image into the wrong direction. My knowledge is very limited, so I dont understand the code completely.
You could do this:
M = 16; N = 20; img1 = randi(255,M,N); % Create a random M x N image
ii = toeplitz(1:N,circshift(fliplr(1:N)',1)); % Create an indexing variable
% Create layers that are shifted copies of the image
array = reshape(img1(:,ii),M,N,N);
As long as your image dimensions don't change, you only ever need to create the ii variable once. After that, you can call the last line each time your image changes. I don't know for sure that this will give you a speed advantage over a for loop, but it is vectorized like you requested. :)
UPDATE
In light of the new information shared about the problem, this solution should give you an order of magnitudes increase in speed:
clear all;
% Set image sizes
M = 360; N = 500;
% Number of column shifts to test
ncols = 200;
% Create comparison matrix (see NOTE)
I = speye(N); E = -1*I;
ii = toeplitz([1:N],[1,N:-1:(N-ncols+2)]);
D = vertcat(repmat(I,1,ncols),E(:,ii));
% Generate some test images
img1 = randi(255,M,N);
img2 = randi(255,M,N);
% Compare images (vectorized)
data_c = reshape(abs([img2,img1]*D),M,N,ncols);
% Compare images (for loop)
array = zeros(M,N,ncols); % <-- Pre-allocate this array!
for i=1:ncols
array(:,:,i)=imabsdiff(circshift(img1,[0 i-1]),img2);
end
This uses matrix multiplication to do the comparisons instead of generating a whole bunch of shifted copies of the image.
NOTE: The matrix D should only be generated one time if your image size is not changing. Notice that the D matrix is completely independent of the images, so it would be wasteful to regenerate it every time. However, if the image size does change, you will need to update D.
Edit: I have updated the code to more closely match what you seem to be looking for. Then I throw the "original" for-loop implementation in to show that they give the same result. One thing worth noting about the vectorized version is that it has the potential to be very memory instensive. If ncols = N then the D matrix has N^3 elements. Even though D is sparse, things fall apart fast when you multiply D by the non-sparse images.
Also, notice that I pre-allocate array before the for loop. This is always good practice in Matlab, where practical, and it will almost invariably give you a large performance boost over the dynamic sizing.
If question is understood correctly, I think you need for loop
for v=1:1:20
array(:,:,v)=circshift(image,[0 v]);
end
I've done quite a bit of searching and haven't been able to find a satisfactory answer so far, so I'm sorry if this question has already been raised.
I'm stuck on how to sum over the dimensions of an array. I have array A(w0,lambda,2048,2048), and I would like to be able to define a second array U(w0, 2048, 2048) which is composed of the sum of A over dimension lambda.
So far I have been defining both A and U as follows:
A = zeros(length(w0),length(lambda),2048,2048);
U = zeros(length(w0),2048,2048);
for ii = 1:length(w0) % Scan through spot sizes
for aa = 1:length(lambda) % Scan through wavelengths
A(ii,aa,:,:) = ASM(w0(ii),lambda(aa),z1,z2);
end
U(ii,:,:) = sum(A,2);
end
Where ASM is just a function. z1 and z2 are defined earlier, and not relevant here.
I have been trying to come up with other possible ways of finding U(w0,2048,2048) as the sum over the second dimension of A (lambda), but haven't been successful...
Thanks for any pointers, and sorry again if this has already been resolved!
James.
From the sounds of it, you just want:
U = squeeze(sum(A, 2));
squeeze() eliminates singleton dimensions.
Here are two alternative solutions:
U = reshape(sum(A, 2), [length(w0) 2048 2048]);
or:
U = sum(A, 2);
U = U(:, 1, :, :);
Try using 'sum' function with a dimension argument, and collapse result on the desired dimensions.
z = rand(2,3,2,2);
q = sum(z,2); %sum all 3 matrices of size 2x2x2 to get 2x1x2x2 result
zz = q(:,1,:,:); %zz is now 2x2x2, by collapsing the dimension 2.