Related
I would like to construct a function
[B, ind] = extract_ones(A)
which removes some sub-arrays from a binary array A in arbitrary dimensions, such that the remaining array B is the largest possible array with only 1's, and I also would like to record in ind that where each of the 1's in B comes from.
Example 1
Assume A is a 2-D array as shown
A =
1 1 0 0 0 1
1 1 1 0 1 1
0 0 0 1 0 1
1 1 0 1 0 1
1 1 0 1 0 1
1 1 1 1 1 1
After removing A(3,:) and A(:,3:5), we have the output B
B =
1 1 1
1 1 1
1 1 1
1 1 1
1 1 1
which is the largest array with only ones by removing rows and columns of A.
As the fifteen 1's of B corresponds to
A(1,1) A(1,2) A(1,6)
A(2,1) A(2,2) A(2,6)
A(4,1) A(4,2) A(4,6)
A(5,1) A(5,2) A(5,6)
A(6,1) A(6,2) A(6,6)
respectively, or equivalently
A(1) A(7) A(31)
A(2) A(8) A(32)
A(4) A(10) A(34)
A(5) A(11) A(35)
A(6) A(12) A(36)
so, the output ind looks like (of course ind's shape does not matter):
ind = [1 2 4 5 6 7 8 10 11 12 31 32 34 35 36]
Example 2
If the input A is constructed by
A = ones(6,3,4,3);
A(2,2,2,2) = 0;
A(4,1,3,3) = 0;
A(1,1,4,2) = 0;
A(1,1,4,1) = 0;
Then, by deleting the minimum cuboids containing A(2,2,2,2), A(4,1,3,3), A(1,1,4,3) and A(1,1,4,1), i.e. after deleting these entries
A(2,:,:,:)
A(:,1,:,:)
Then the remaining array B will be composed by 1's only. And the ones in B corresponds to
A([1,3:6],2:3,1:4,1:3)
So, the output ind lists the subscripts transformed into indices, i.e.
ind = [7 9 10 11 12 13 15 16 17 18 25 27 28 29 30 31 33 34 35 36 43 45 46 47 48 49 51 52 53 54 61 63 64 65 66 67 69 70 71 72 79 81 82 83 84 85 87 88 89 90 97 99 100 101 102 103 105 106 107 108 115 117 118 119 120 121 123 124 125 126 133 135 136 137 138 139 141 142 143 144 151 153 154 155 156 157 159 160 161 162 169 171 172 173 174 175 177 178 179 180 187 189 190 191 192 193 195 196 197 198 205 207 208 209 210 211 213 214 215 216]
As the array needed to be processed as above is in 8-D, and it should be processed more than once, so can anyone give me opinions on how to composing the program doing this task fast?
My work so far [Added at 2 am (GMT-4), 2nd Aug 2017]
My idea was that I delete the sub-arrays with the largest proportion of zero one by one. And here is my work so far:
Inds = reshape(1:numel(A),size(A)); % Keep track on which 1's survive.
cont = true;
while cont
sz = size(A);
zero_percentage = 0;
Test_location = [];
% This nested for loops are for determining which sub-array of A has the
% maximum proportion of zeros.
for J = 1 : ndims(A)
for K = 1 : sz(J)
% Location is in the form of (_,_,_,...,_)
% where the J-th blank is K, the other blanks are colons.
Location = strcat('(',repmat(':,',1,(J-1)),int2str(K),repmat(',:',1,(ndims(A)-J)),')');
Test_array = eval(strcat('A',Location,';'));
N = numel(Test_array);
while numel(Test_array) ~= 1
Test_array = sum(Test_array);
end
test_zero_percentage = 1 - (Test_array/N);
if test_zero_percentage > zero_percentage
zero_percentage = test_zero_percentage;
Test_location = Location;
end
end
end
% Delete the array with maximum proportion of zeros
eval(strcat('A',Test_location,'= [];'))
eval(strcat('Inds',Test_location,'= [];'))
% Determine if there are still zeros in A. If there are, continue the while loop.
cont = A;
while numel(cont) ~= 1
cont = prod(cont);
end
cont = ~logical(cont);
end
But I encountered two problems:
1) It may be not efficient to check all arrays in all sub-dimensions one-by-one.
2) The result does not contain the most number of rectangular ones. for example, I tested my work using a 2-dimensional binary array A
A =
0 0 0 1 1 0
0 1 1 0 1 1
1 0 1 1 1 1
1 0 0 1 1 1
0 1 1 0 1 1
0 1 0 0 1 1
1 0 0 0 1 1
1 0 0 0 0 0
It should return me the result as
B =
1 1
1 1
1 1
1 1
1 1
1 1
Inds =
34 42
35 43
36 44
37 45
38 46
39 47
But, instead, the code returned me this:
B =
1 1 1
1 1 1
1 1 1
Inds =
10 34 42
13 37 45
14 38 46
*My work so far 2 [Added at 12noon (GMT-4), 2nd Aug 2017]
Here is my current amendment. This may not provide the best result.
This may give a fairly OK approximation to the problem, and this does not give empty Inds. But I am still hoping that there is a better solution.
function [B, Inds] = Finding_ones(A)
Inds = reshape(1:numel(A),size(A)); % Keep track on which 1's survive.
sz0 = size(A);
cont = true;
while cont
sz = size(A);
zero_percentage = 0;
Test_location = [];
% This nested for loops are for determining which sub-array of A has the
% maximum proportion of zeros.
for J = 1 : ndims(A)
for K = 1 : sz(J)
% Location is in the form of (_,_,_,...,_)
% where the J-th blank is K, the other blanks are colons.
Location = strcat('(',repmat(':,',1,(J-1)),int2str(K),repmat(',:',1,(ndims(A)-J)),')');
Test_array = eval(strcat('A',Location,';'));
N = numel(Test_array);
Test_array = sum(Test_array(:));
test_zero_percentage = 1 - (Test_array/N);
if test_zero_percentage > zero_percentage
eval(strcat('Testfornumel = numel(A',Location,');'))
if Testfornumel < numel(A) % Preventing the A from being empty
zero_percentage = test_zero_percentage;
Test_location = Location;
end
end
end
end
% Delete the array with maximum proportion of zeros
eval(strcat('A',Test_location,'= [];'))
eval(strcat('Inds',Test_location,'= [];'))
% Determine if there are still zeros in A. If there are, continue the while loop.
cont = A;
while numel(cont) ~= 1
cont = prod(cont);
end
cont = ~logical(cont);
end
B = A;
% command = 'i1, i2, ... ,in'
% here, n is the number of dimansion of A.
command = 'i1';
for J = 2 : length(sz0)
command = strcat(command,',i',int2str(J));
end
Inds = reshape(Inds,numel(Inds),1); %#ok<NASGU>
eval(strcat('[',command,'] = ind2sub(sz0,Inds);'))
% Reform Inds into a 2-D matrix, which each column indicate the location of
% the 1 originated from A.
Inds = squeeze(eval(strcat('[',command,']')));
Inds = reshape(Inds',length(sz0),numel(Inds)/length(sz0));
end
It seems a difficult problem to solve, since the order of deletion can change a lot in the final result. If in your first example you start with deleting all the columns that contain a 0, you don't end up with the desired result.
The code below removes the row or column with the most zeros and keeps going until it's only ones. It keeps track of the rows and columns that are deleted to find the indexes of the remaining ones.
function [B,ind] = extract_ones( A )
if ~islogical(A),A=(A==1);end
if ~any(A(:)),B=[];ind=[];return,end
B=A;cdel=[];rdel=[];
while ~all(B(:))
[I,J] = ind2sub(size(B),find(B==0));
ih=histcounts(I,[0.5:1:size(B,1)+0.5]); %zero's in rows
jh=histcounts(J,[0.5:1:size(B,2)+0.5]); %zero's in columns
if max(ih)>max(jh)
idxr=find(ih==max(ih),1,'first');
B(idxr,:)=[];
%store deletion
rdel(end+1)=idxr+sum(rdel<=idxr);
elseif max(ih)==max(jh)
idxr=find(ih==max(ih),1,'first');
idxc=find(jh==max(jh),1,'first');
B(idxr,:)=[];
B(:,idxc)=[];
%store deletions
rdel(end+1)=idxr+sum(rdel<=idxr);
cdel(end+1)=idxc+sum(cdel<=idxc);
else
idxc=find(jh==max(jh),1,'first');
B(:,idxc)=[];
%store deletions
cdel(end+1)=idxc+sum(cdel<=idxc);
end
end
A(rdel,:)=0;
A(:,cdel)=0;
ind=find(A);
Second try: Start with a seed point and try to grow the matrix in all dimensions. The result is the start and finish point in the matrix.
function [ res ] = seed_grow( A )
if ~islogical(A),A=(A==1);end
if ~any(A(:)),res={};end
go = true;
dims=size(A);
ind = cell([1 length(dims)]); %cell to store find results
seeds=A;maxmat=0;
while go %main loop to remove all posible seeds
[ind{:}]=find(seeds,1,'first');
S = [ind{:}]; %the seed
St = [ind{:}]; %the end of the seed
go2=true;
val_dims=1:length(dims);
while go2 %loop to grow each dimension
D=1;
while D<=length(val_dims) %add one to each dimension
St(val_dims(D))=St(val_dims(D))+1;
I={};
for ct = 1:length(S),I{ct}=S(ct):St(ct);end %generate indices
if St(val_dims(D))>dims(val_dims(D))
res=false;%outside matrix
else
res=A(I{:});
end
if ~all(res(:)) %invalid addition to dimension
St(val_dims(D))=St(val_dims(D))-1; %undo
val_dims(D)=[]; D=D-1; %do not try again
if isempty(val_dims),go2=false;end %end of growth
end
D=D+1;
end
end
%evaluate the result
mat = prod((St+1)-S); %size of matrix
if mat>maxmat
res={S,St};
maxmat=mat;
end
%tried to expand, now remove seed option
for ct = 1:length(S),I{ct}=S(ct):St(ct);end %generate indices
seeds(I{:})=0;
if ~any(seeds),go=0;end
end
end
I tested it using your matrix:
A = [0 0 0 1 1 0
0 1 1 0 1 1
1 0 1 1 1 1
1 0 0 1 1 1
0 1 1 0 1 1
0 1 0 0 1 1
1 0 0 0 1 1
1 0 0 0 0 0];
[ res ] = seed_grow( A );
for ct = 1:length(res),I{ct}=res{1}(ct):res{2}(ct);end %generate indices
B=A(I{:});
idx = reshape(1:numel(A),size(A));
idx = idx(I{:});
And got the desired result:
B =
1 1
1 1
1 1
1 1
1 1
1 1
idx =
34 42
35 43
36 44
37 45
38 46
39 47
This question already has an answer here:
Content-Based Image Retrieval and Precision-Recall graphs using Color Histograms in MATLAB
(1 answer)
Closed 7 years ago.
I want to implement the following Matlab function:
function hist = binnedRgbHist(im, numChannelBins)
Given an image im and a number between 1 and 256 numChannelBins, it should create a histogram sized (numChannelBins)^3.
For example, if numChannelBins is 2, it should produce the following 8-sized histogram:
Number of pixels with R < 128, G < 128, B < 128
Number of pixels with R < 128, G < 128, B >= 128
Number of pixels with R < 128, G >= 128, B < 128
Number of pixels with R < 128, G >= 128, B >= 128
Number of pixels with R > 128, G < 128, B < 128
Number of pixels with R > 128, G < 128, B >= 128
Number of pixels with R > 128, G >= 128, B < 128
Number of pixels with R > 128, G >= 128, B >= 128
It is like creating a cube where each axis represents one of (R,G and B), where each axis is divided into 2 bins => Finally there are 8 bins in the cube.
My questions:
It there a built-in function for it?
If not, how is it better to implement it in manners of runtinme using the GPU? Should I better iterate over the pixels once and create the histogram manually, or should I better iterate over the bins and each time count the number of pixels which satisfy the bin's conditions?
accumarray is very suited for this. Let
im: input image;
N: number of bins per color component.
Then
result = accumarray(reshape(permute(ceil(im/255*N), [3 1 2]), 3, []).', 1, [N N N]);
How it works
ceil(im/255*N) quantizes each color vaue to 1, 2, ..., N.
reshape(permute(..., [3 1 2]), 3, []).' transforms the quantized image into a three-column matrix where each row is a pixel and each column is a (quantized) color component.
accumarray(..., 1, [N N N]) considers each row of that matrix as 3D index, and counts how many times each index appears, giving filling indices that don't appear with a 0.
Example 1
Data:
>> N = 2;
>> im = randi(256,4,5,3)
im(:,:,1) =
113 152 157 65 229
138 71 215 39 41
13 108 230 160 153
142 128 125 220 214
im(:,:,2) =
208 215 182 27 230
205 161 8 95 180
225 53 73 129 31
103 97 160 83 255
im(:,:,3) =
242 29 185 89 55
202 225 156 174 96
160 197 35 87 113
244 176 146 85 120
Result:
result(:,:,1) =
1 1
3 4
result(:,:,2) =
2 4
3 2
It can be checked for example that there is only 1 pixel with all R,G,B less than 128.
Example 2
Data:
>> im = repmat(150,20,30,3);
>> N = 4;
Result:
result(:,:,1) =
0 0 0 0
0 0 0 0
0 0 0 0
0 0 0 0
result(:,:,2) =
0 0 0 0
0 0 0 0
0 0 0 0
0 0 0 0
result(:,:,3) =
0 0 0 0
0 0 0 0
0 0 600 0
0 0 0 0
result(:,:,4) =
0 0 0 0
0 0 0 0
0 0 0 0
0 0 0 0
In this case all pixels belong to the same 3D-bin:
I see #Luis Mendo gave a great one-liner solution as I was writing this. In case it provides some deeper intuition, my solution makes use of histcounts and accumarray:
im = randi([1 255],[10,5,3]); %// A random 10-by-5 "image"
numChannelBins = 2;
[~,~,binR]=histcounts(im(:,:,1),[1 ceil((1:numChannelBins)*(255/numChannelBins))]);
[~,~,binG]=histcounts(im(:,:,2),[1 ceil((1:numChannelBins)*(255/numChannelBins))]);
[~,~,binB]=histcounts(im(:,:,3),[1 ceil((1:numChannelBins)*(255/numChannelBins))]);
hist=accumarray([binR(:) binG(:) binB(:)],1,[numChannelBins,numChannelBins,numChannelBins])
Explanation:
the three calls to histcounts bin the red, green, blue pixels separately -- the third output [~,~,binX] of histcounts gives the bin index for each pixel
accumarray accumulates all the unique index triplets
I need to make a handwritten image to be tested with a neural network in Matlab. When I see the data contained in the training images from the MNIST I see that it is an array of different gray scales like:
Columns 343 through 351
0 -0.0240 0.4002 0.6555 0.0235 -0.0062 0 0 0
Columns 352 through 360
0 0 0 0 0 0 0 0 0
Columns 361 through 369
0 0 0 -0.0079 0.1266 0.3272 -0.0233 0.0005
corresponding to a 20x20 image, unrolled into a 1*400 dimensional array.
I have downloaded an image in jpeg format and did the following:
im=imread('image.jpg');
gi=rgb2gray(im);
gi=gi(:);
gi=gi';
that generates me an array gi that says <1*400 uint8>, the last part of uint8 does not appear in the MNIST samples when I put it in Matlab. When I check it up my array it appear the following values:
Columns 289 through 306
58 105 128 133 142 131 76 21 1 0 3 0 2 4 17 12 7 0
Columns 307 through 324
1 15 42 75 97 105 98 73 31 4 1 0 0 0 0 2 4 3
Columns 325 through 342
0 0 1 4 21 37 55 59 46 26 9 0 0 0 0 0 0 0
Columns 343 through 360
1 1 0 0 0 1 7 14 21 21 14 5 0 0 0 0 0 0
Columns 361 through 378
0 0 0 0 0 0 0 0 0 1 2 1 0 0 0 2 0 0
when I visualize them all is fine, but when I want to run my program the following message appears:
??? Error using ==> mtimes
MTIMES is not fully supported for integer classes. At least one input must be scalar.
Error in ==> predict at 15
h1 = sigmoid([ones(m, 1) X] * Theta1');
Error in ==> ex4 at 241
pred = predict(Theta1, Theta2, gi);
situation that does not occur when I test my program even with one random sample ofc the MNIST data; any help?
You could try something like this:
imfile = 'image.jpg';
im = double(rgb2gray(imread(imfile))); % double and convert to grayscale
im = imresize(im,[20,20]); % change to 20 by 20 dimension
im = im(:); % unroll matrix to vector
im = im./max(im);
Note the MNIST dataset is intended to be a good dataset to require minimal preprocessing and the images were actually originally black and white (bilevel) whereas you are using color image. Also they do normalisation and other preprocessing to make nice 28 by 28 image dataset, my brief snippet of code above is unlikely to be anywhere near as good as MNIST dataset and is just intended to attempt to fix your error.
Your specific error is likely because you don't use double().
You may also get further errors because your code needs right dimensions, which can be achieved using imresize.
More information on MNIST dataset here:
http://yann.lecun.com/exdb/mnist/
I have written a routine to give the column index position for the furthers right cell containing a 1, marking the right edge of a polygon in a mask array.
But when I print the index arrays (I realize the row position array is redundant), all rows are reporting a position (other than 0), which I know shouldn't be the case. I can't seem to find where the following could be incorrect.
Example mask array:
0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 1 0 0 0 0 0 0
0 0 0 0 0 0 1 1 1 0 0 0 0 0
0 0 0 0 0 1 1 1 1 1 1 0 0 0
0 0 0 1 1 1 1 1 1 1 1 0 0 0
0 0 0 1 1 1 1 1 1 1 1 0 0 0
0 0 0 0 0 0 1 1 1 1 0 0 0 0
0 0 0 0 0 0 0 0 0 1 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0
Desired output:
[0,10,10,11,11,11,9,8,0]
Is this a Fortran thing, or is my logic just off (
Routine:
subroutine get_right_idx(mask_array, idx_x, idx_y)
integer, parameter :: x = 169 ! num columns
integer, parameter :: y = 124 ! num rows
integer i ! iterator for rows
integer j ! iterator for columns
integer row_x ! x position for furthest south cell in row
integer row_y ! y position for furthest south cell in row
integer :: idx_x(y) ! index positions for lowest lat in model grid - x
integer :: idx_y(y) ! index positions for lowest lat in model grid - y
real mask_array(y,x) ! mask array of zeros, containing polygon of ones
do j=1,y
row_x = 0
row_y = 0
do i=1,x
if (mask_array(j,i).eq.1) then
row_x = i
row_y = j
endif
enddo
idx_x(j)=row_x
idx_y(j)=row_y
enddo
endsubroutine get_right_idx
Actual mask (zoomed out in Open Office):
Below is the mask that I am trying to evaluate. My resulting array has a non-zero value in for all elements, where there should be zero elements at the start and end of the array, no matter which direction it is evaluated from.
Output:
125 104 104 104 104 104 104 114 114 114 114 103 103 103 108 108 103 103 103 103 97 97 97 107 107 107 107 107 107 107 107 107 97 101 101 101 101 101 111 111 111 111 111 111 101 101 100 105 105 105 105 105 105 100 115 115 104 104 104 104 104 104 104 104 104 104 98 98 98 98 108 108 108 108 108 108 108 108 98 102 102 102 102 102 112 112 112 112 112 112 101 101 101 106 106 106 101 101 101 95 95 105 105 105 105 105 105 105 105 105 105 99 99 99 99 99 109 109 109 109 109 109 109 99 99
I made a quick template of your code, but with my suggestion. Since you want to find the largest index j that has a 1 in it, you can simply test this by starting your j-loop with the maximum (in the example, 14) and working downwards but exiting the inner do-loop if mask_array==1.
You could generalize this by replacing 14 and 9 with the maximum values for those dimensions.
program get_right
implicit none
integer, dimension(9,14) :: mask_array
integer :: i,j,mask_out(9),j_tmp
mask_array(1,:)=[0,0,0,0,0,0,0,0,0,0,0,0,0,0]
mask_array(2,:)=[0,0,0,0,0,0,0,1,0,0,0,0,0,0]
mask_array(3,:)=[0,0,0,0,0,0,1,1,1,0,0,0,0,0]
mask_array(4,:)=[0,0,0,0,0,1,1,1,1,1,1,0,0,0]
mask_array(5,:)=[0,0,0,1,1,1,1,1,1,1,1,0,0,0]
mask_array(6,:)=[0,0,0,1,1,1,1,1,1,1,1,0,0,0]
mask_array(7,:)=[0,0,0,0,0,0,1,1,1,1,0,0,0,0]
mask_array(8,:)=[0,0,0,0,0,0,0,0,0,1,0,0,0,0]
mask_array(9,:)=[0,0,0,0,0,0,0,0,0,0,0,0,0,0]
mask_out=0
do i=1,9
j_tmp=0
print '(14(i0,2x))',mask_array(i,:)
do j=14,1,-1
if(mask_array(i,j)==1) then
j_tmp=j
exit
endif
enddo
mask_out(i)=j
enddo
print *,""
print '(9(i0,2x))',mask_out
end program get_right
I got 0,8,9,11,11,11,10,10,0 as the answer (which is what you got but backwards).
Or, if typing makes you tired you could evaluate this
maxval(mask_array*spread([(ix,ix=lbound(mask_array,dim=2),&
ubound(mask_array,dim=2))],dim=1, ncopies=size(mask_array,dim=1)),dim=2)
As you can see this makes a temporary array (which may be undesirable if your masks are large) using spread and an implied-do loop, each element of this temporary contains its own column index number. Then it multiplies the temporary with the mask_array, performing element-wise multiplication not matrix multiplication. Finally take the maxval of each row of the result. This returns the vector in the same order as Kyle's code does.
I've edited the code to use lbound and ubound rather than 1 and size in case you want to use the code on arrays with lower bounds other than 1.
Bob's yer uncle, but don't ask whether this is faster than Kyle's code. If you are interested in execution speed test and measure.
Incidentally, since this returns just one vector and doesn't modify its arguments or have any other side effects, I'd package it as a function rather than as a subroutine.
I want to assign the same id number to every four observations. For example, if I have the following data
age marital gender id
45 1 0 1
33 1 1 1
68 0 1 1
27 1 0 1
43 0 0 2
37 0 1 2
19 1 1 2
40 1 1 2
25 1 0 3
38 1 1 3
57 0 0 3
50 1 0 3
51 1 1 4
44 0 1 4
69 1 0 4
39 0 1 4
The last column id is something I want to produce.
Plus, the dataset have 500,000+ observations.
Thanks in advance.
Slightly more compact:
id = ceil(_n_/4);
Use the integer function and the built-in _n_ variable (which increments for each observation):
id = int( (_n_-4)/4 )+1;