Related
I'm wondering if there's an indexable way of doing the following code on Octave, as it's iterative and thus really slow compared to working with indexation.
for i = [1:size(A, 1)]
for j = [1:size(A, 2)]
if (max(A(i, j, :)) == 0)
A(i, j, :) = B(i, j, :);
endif
endfor
endfor
A and B are two RGB images that overlaps and I want A(i,j) to have B(i,j) value if A(i,j) is 0 on all of the three channels. It is very slow in this form but I'm not experimented enough with this language to vectorize it.
Your code can be vectorized as follows:
I = max(A,[],3) == 0;
I = repmat(I,1,1,3);
A(I) = B(I);
The first line is a direct copy of your max conditional statement within the loop, but vectorized across all of A. This returns a 2D array, which we cannot directly use to index into the 3D arrays A and B, so we apply repmat to replicate it along the 3rd dimension (the 3 here is the number of repetitions, we're assuming A and B are RGB images with 3 elements along the 3rd dimension). Finally, an indexed assignment copies the relevant values over from B to A.
To generalize this to any array size, replace the "3" in the repmat statement with size(A,3).
Not adding much here, but perhaps this will give you a better understanding so worth adding another solution.
% example data
A = randi( 255, [2,4,3] ); A(2,2,:) = [0,0,0];
B = randi( 255, [2,4,3] );
% Logical array with size [Dim1, Dim2], such that Dim3 is 'squashed' into a
% single logical value at each position, indicating whether the third dimension
% at that position does 'not' have 'any' true (i.e. nonzero) values.
I = ~any(A, 3);
% Use this to index A and B for assignment.
A([I,I,I]) = B([I,I,I])
This approach may be more efficient than the repmat one, which is a slightly more expensive operation, but may be slightly less obvious to understand why it works. But. Understanding how this works teaches you something about matlab/octave, so it's a nice learning point.
Matlab and Octave store arrays in column major order (as opposed to, say, Python). This is also the reason that doing A(:) will return A as a vector, constructed in a column-by-column basis. It is also the reason that you can index a 3-dimensional array using a single index (called a "linear index"), which will correspond to the element you reach when you count that number of elements going down columns.
When performing logical indexing, matlab/octave effectively takes a logical vector, matches each linear index of that vector to the equivalent linear index of A and decides whether to return it or not, based on whether the boolean value of the logical index at that linear index is true or false. If you provide a logical array I that is of a smaller size than A, the indexing will simply stop at the last linear index of I. Specifically, note that the shape of I is irrelevant, since it will be interpreted in a linear indexing manner anyway.
In other words, logical indexing with I is the same as logical indexing with I(:), and logical indexing with [I,I,I] is the same as logical indexing with [ I(:); I(:); I(:) ].
And if I is of size A(:,:,1) then [I,I,I] is of size A(:,:,:), such that in a linear indexing sense it can be used as a valid logical index matching each linear index of I to the equivalent linear index of A.
The max() function can take a single matrix and return the maximum value along a dimension
There's also the all() function that tells you if all values along a dimension are nonzero, and the any() function that tells you if any of the values along a dimension are nonzero
A = reshape(1:75, 5, 5, 3)
A(2, 3, :) = 0;
B = ones(size(A)) * 1000
use_pixel_from_A = any(A, 3)
use_pixel_from_B = ~use_pixel_from_A
Now for each element of the 3rd axis, you know which pixels to take from A and which to take from B. Since our use_pixel... matrices contain 0 and 1, we can element-wise multiply them to A and B to filter out elements of A and B as required.
C = zeros(size(A));
for kk = 1:size(A, 3)
C(:, :, kk) = A(:, :, kk) .* use_pixel_from_A + B(:, :, kk) .* use_pixel_from_B
end
I'm looking to find a way to take an array in ruby, two indices in that array and return an enumerable object which will yield, in order, all the elements between and including the two indices. But for performance reasons, I want to do this subject to the following two conditions:
This slice to enum does not create a copy of the subarray I want a return an enum over. This rules out array[i..j].to_enum, for example because array[i..j] is creating a new array.
It's not necessary to loop over the entire array to create the enum.
I'm wondering if there's a way to do this using the standard library's enumerable or array functionality without having to explicitly create my own custom enumerator.
What I'm looking for is a cleaner way to create the below enumerator:
def enum_slice(array, i, j)
Enumerator.new do |y|
while i <= j
y << array[i] # this is confusing syntax for yield (see here: https://ruby-doc.org/core-2.6/Enumerator.html#method-c-new)
i += 1
end
end
end
That seems pretty reasonable, and could even be turned into an extension to Array itself:
module EnumSlice
def enum_slice(i, j)
Enumerator.new do |y|
while i <= j
y << self[i]
i += 1
end
end
end
end
Now within the Enumerator block, y represents a Proc you call when you have more data. If that block ends it's presumed you're done enumerating. There's no requirement to ever terminate, an infinite Enumerator is allowed, and in that case it's up to the caller to stop iterating.
So in other words, the y block argument can be called zero or more times, and each time it's called output is "emitted" from the enumerator. When that block exits the enumerator is considered done and is closed out, y is invalid at that point.
All y << x does is call the << method on Enumerator::Yielder, which is a bit of syntactical sugar to avoid having to do y.call(x) or y[x], both of which look kind of ugly.
Now you can add this to Array:
Array.include(EnumSlice)
Where now you can do stuff like this:
[ 1, 2, 3, 4, 5, 6 ].enum_slice(2, 4).each do |v|
p v
end
Giving you the correct output.
It's worth noting that despite having gone through all this work, this really doesn't save you any time. There's already built-in methods for this. Your enum_slice(a, i, j) method is equivalent to:
a.drop(i).take(j)
Is that close in terms of performance? A a quick benchmark can help test that theory:
require 'benchmark'
Benchmark.bm do |bm|
count = 10000
a = (0..100_000).to_a
bm.report(:enum_slice) do
count.times do
a.enum_slice(50_000, 25_000).each do
end
end
end
bm.report(:drop_take) do
count.times do
a.drop(50_000).take(25_000).each do
end
end
end
end
The results are:
user system total real
enum_slice 0.020536 0.000200 0.020736 ( 0.020751)
drop_take 7.682218 0.019815 7.702033 ( 7.720876)
So your approach is about 374x faster. Not bad!
I have a fortran array a(i,j). I wish to sum it on dimension 2(j) with a mask that j is not equal to i.
i.e,
a1=0
do j=1,n
if(j.ne.i) then
a1=a1+a(i,j)
endif
enddo
What is the way of doing this using the intrinsic sum function in fortran as I found the intrinsic to be much faster than the explicit loop.
I thought of trying sum(a(i,:),j.ne.i), but this is naturally giving error. Also if one can suggest how to only some the values of a(i,:) where abs(a(i,j)) is greater than, say 0.01, it would be helpful.
You can easily avoid any branching for the off-diagonal case. It should be much faster than creating any mask array and checking the mask. Branching (conditional jumps) is costly even when branch prediction can be very efficient.
do j=1,n
do i = 1,j-1
a1=a1+a(i,j)
end do
do i = j+1,n
a1=a1+a(i,j)
end do
end do
If you need your code to be fast and not short, you should test this kind of approach. In my tests it is much faster.
To answer your last question, you can use the WHERE construct to build a mask. For example,
logical :: y(3,3) = .false.
real x(3,3)
x = 1
x(1,1) = 0.1
x(2,2) = 0.1
x(3,3) = 0.1
print * , sum(x)
where(abs(x) > 0.25) y = .true.
print *, sum(x,y)
end
Whether this is better than nested do-loops is questionable.
I find that summing the whole array then subtracting sum of diagonal elements can be 2x faster.
a1 = 0
do i = 1, n
a1 = a1 + a(i,i)
end do
a1 = sum(a) - a1
end do
I have a very long one-dimensional array of positive integers. Starting from one end, I need to find the longest slices/chunks of the array that have values that are at-least one number away from all the constituents of that slice.
i.e., I want to make a partitioning of the array (starting from the left) such that each partition contains elements that are more than one unit away from all the other elements in that partition.
Eg:
[1,1,9,5,3,8,7,4,1,2] -> [1],[1,9,5,3],[8],[7,4,1],[2]
[1,5,9,1,3,6,4,2,7,0] -> [1,5,9],[1,3,6,4],[2,7,0]
Bellow, I've written a little code in Fortran that will let me find the first such point of recurrence of a previous value.
mask is a LOGICAL array
array is the array in question
n is the length of the array
I can easily extend this to find the full partitioning.
mask = .FALSE.
DO i = 1,n
k = array(i)
IF ( mask(k) ) THEN
PRINT*, i
EXIT
ELSE
mask(k-1 : k+1) = .TRUE.
END IF
END DO
So my question is, is there a better way (alorithm) to do this? When I say better, I mean speed. I don't mind a memory cost.
Conceptually it could look something like this...
DO i = 1,n-1
Delta(I) = array(I+1) - array(I)
ENDDO
iMask = 0
WHERE(ABS(Delta) < 2) iMask =1
ALLOCATE(splits(SUM(iMask)))
K=0
DO I = 1, n-1
IF(iMask(I) == 0) CYCLE
K = K +1
Splits(K) = I
ENDDO
!... DEALLOCATE(Splits)
Then just print out the data between the splits values, which could be off by a count, and you may also need to do something for the Nth point, so it depends a bit on your implementation and whether your delta is "too the next point" or "from the last point".
In this case I used imask as an integer rather than a logical so I could use SUM.
My initial reaction is the naive approach:
save index bounds on the partition you're currently expanding (partitionNumber from iStart to iEnd)
Take the next point with index iEnd+1 and loop from iStart to iEnd testing that the candidate point is not within 1 of the current members
If the candidate fails the inclusion test, start it in its own partition by resetting iStart and incrementing partitionNumber
Increment iEnd.
If you're expecting the partitions to mostly be quite short then this should be pretty quick. If you're expecting long chains of increasing or decreasing integers you could save the min and max of values in the partition include a quick test to see if your candidate is outside the range.
I've not tested this and my fortran might be a bit rusty, but I think it represents the above algorithm.
partitionNumber = 1
iStart = 1
iEnd = 1
iCandidate = iEnd + 1
arrayMember(iStart) = partitionNumber
DO WHILE (iCandidate <= N)
DO j = iStart,iEnd
IF ( ABS(array(iCandidate)-array(j)) < 2 )
partitionNumber = partitionNumber + 1
iStart = iCandidate
EXIT
END IF
END DO
arrayMember(iCandidate) = partitionNumber
iEnd = iEnd + 1
iCandidate = iEnd + 1
END DO
Operating on your two examples, I would hope it to return arrayMember with entries
[1,1,9,5,3,8,7,4,1,2] -> [1,2,2,2,2,3,4,4,4,5] (represents [1],[1,9,5,3],[8],[7,4,1],[2])
[1,5,9,1,3,6,4,2,7,0] -> [1,1,1,2,2,2,3,3,3,3] (represents [1,5,9],[1,3,6],[4,2,7,0])
I'm not entirely sure I understand how you would extend your version to all partitions, but this might save on defining mask of size MAX(array)?
I am using Matlab for one of my projects. I am actually stuck at a point since some time now. Tried searching on google, but, not much success.
I have an array of 0s and 1s. Something like:
A = [0,0,0,1,1,1,1,1,0,0,1,1,1,1,1,1,0,0,0,0,0,1,1,1,0,0,0,0];
I want to extract an array of indicies: [x_1, x_2, x_3, x_4, x_5, ..]
Such that x_1 is the index of start of first range of zeros. x_2 is the index of end of first range of zeros.
x_3 is the index of start of second range of zeros. x_4 is the index of end of second range of zeros.
For the above example:
x_1 = 1, x_2 = 3
x_3 = 9, x_4 = 10
and so on.
Of course, I can do it by writing a simple loop. I am wondering if there is a more elegant (vectorized) way to solve this problem. I was thinking about something like prefix some, but, no luck as of now.
Thanks,
Anil.
The diff function is great for this sort of stuff and pretty quick.
temp = diff(A);
Starts = find([A(1) == 0, temp==-1]);
Ends = find([temp == 1,A(end)==0])
Edit: Fixed the error in the Ends calculation caught by gnovice.
Zeros not preceded by other zeros: A==0 & [true A(1:(end-1))~=0]
Zeros not followed by other zeros: A==0 & [A(2:end)~=0 true]
Use each of these plus find to get starts and ends of runs of zeros. Then, if you really want them in a single vector as you described, interleave them.
If you want to get your results in a single vector like you described above (i.e. x = [x_1 x_2 x_3 x_4 x_5 ...]), then you can perform a second-order difference using the function DIFF and find the points greater than 0:
x = find(diff([1 A 1],2) > 0);
EDIT:
The above will work for the case when there are at least 2 zeroes in every string of zeroes. If you will have single zeroes appearing in A, the above can be modified to handle them like so:
diffA = diff([1 A 1],2);
[~,x] = find([diffA > 0; diffA == 2]);
In this case, a single zero value will create repeated indices in x (i.e. if A starts with a single zero, then x(1) and x(2) will both be 1).