What are the reasons for this benchmark result? - loops

Question

Two functions that convert a rgb image to a gray scale image:
function rgb2gray_loop{T<:FloatingPoint}(A::Array{T,3})
r,c = size(A)
gray = similar(A,r,c)
for i = 1:r
for j = 1:c
#inbounds gray[i,j] = 0.299*A[i,j,1] + 0.587*A[i,j,2] + 0.114 *A[i,j,3]
end
end
return gray
end
And:
function rgb2gray_vec{T<:FloatingPoint}(A::Array{T,3})
gray = similar(A,size(A)[1:2]...)
gray = 0.299*A[:,:,1] + 0.587*A[:,:,2] + 0.114 *A[:,:,3]
return gray
end
The first one is using loops, while the second one uses vectorization.
When benchmarking them (with the Benchmark package) I get the following results for different sized input images (f1 is the loop version, f2 the vectorized version):
A = rand(50,50,3):
| Row | Function | Average | Relative | Replications |
|-----|----------|-------------|----------|--------------|
| 1 | "f1" | 3.23746e-5 | 1.0 | 1000 |
| 2 | "f2" | 0.000160214 | 4.94875 | 1000 |
A = rand(500,500,3):
| Row | Function | Average | Relative | Replications |
|-----|----------|------------|----------|--------------|
| 1 | "f1" | 0.00783007 | 1.0 | 100 |
| 2 | "f2" | 0.0153099 | 1.95527 | 100 |
A = rand(5000,5000,3):
| Row | Function | Average | Relative | Replications |
|-----|----------|----------|----------|--------------|
| 1 | "f1" | 1.60534 | 2.56553 | 10 |
| 2 | "f2" | 0.625734 | 1.0 | 10 |
I expected one function to be faster than the other (maybe f1 because of the inbounds macro).
But I can't explain, why the vectorized version gets faster for larger images.
Why is that?

Answer 1

The answer for the results is that multidimensional arrays in Julia are stored in column-major order. See Julias Memory Order.
Fixed looped version, regarding column-major-order (inner and outer loop variables swapped):
function rgb2gray_loop{T<:FloatingPoint}(A::Array{T,3})
r,c = size(A)
gray = similar(A,r,c)
for j = 1:c
for i = 1:r
#inbounds gray[i,j] = 0.299*A[i,j,1] + 0.587*A[i,j,2] + 0.114 *A[i,j,3]
end
end
return gray
end
New results for A = rand(5000,5000,3):
| Row | Function | Average | Relative | Replications |
|-----|----------|----------|----------|--------------|
| 1 | "f1" | 0.107275 | 1.0 | 10 |
| 2 | "f2" | 0.646872 | 6.03004 | 10 |
And the results for smaller Arrays:
A = rand(500,500,3):
| Row | Function | Average | Relative | Replications |
|-----|----------|------------|----------|--------------|
| 1 | "f1" | 0.00236405 | 1.0 | 100 |
| 2 | "f2" | 0.0207249 | 8.76671 | 100 |
A = rand(50,50,3):
| Row | Function | Average | Relative | Replications |
|-----|----------|-------------|----------|--------------|
| 1 | "f1" | 4.29321e-5 | 1.0 | 1000 |
| 2 | "f2" | 0.000224518 | 5.22961 | 1000 |

Answer 2

Just speculation because I don't know Julia-Lang:
I think the statement gray = ... in the vectorized form creates a new Array where all the calculated values are stored, while the old array is scrapped. In f1 the values are overwritten in place, so no new memory allocation is needed. Memory allocation is quite expensive so the loop-version with in-place overwrites is faster for low numbers.
But memory allocation is usually a static overhead (allocation twice as much doesn't take twice as long) and the vectorized version is computing faster (maybe in parallel ?) so if the numbers get big enough the faster calculation makes more difference than the memory allocation.

Answer 3

I cannot reproduce your results.
See this IJulia notebook: http://nbviewer.ipython.org/urls/gist.githubusercontent.com/anonymous/24c17478ae0f5562c449/raw/8d5d32c13209a6443c6d72b31e2459d70607d21b/rgb2gray.ipynb
The numbers I get are:
In [5]:
#time rgb2gray_loop(rand(50,50,3));
#time rgb2gray_vec(rand(50,50,3));
elapsed time: 7.591e-5 seconds (80344 bytes allocated)
elapsed time: 0.000108785 seconds (241192 bytes allocated)
In [6]:
#time rgb2gray_loop(rand(500,500,3));
#time rgb2gray_vec(rand(500,500,3));
elapsed time: 0.021647914 seconds (8000344 bytes allocated)
elapsed time: 0.012364489 seconds (24001192 bytes allocated)
In [7]:
#time rgb2gray_loop(rand(5000,5000,3));
#time rgb2gray_vec(rand(5000,5000,3));
elapsed time: 0.902367223 seconds (800000440 bytes allocated)
elapsed time: 1.237281103 seconds (2400001592 bytes allocated, 7.61% gc time)
As expected, the looped version is faster for large inputs. Also note how the vectorized version allocated three times as much memory.
I also want to point out that the statement gray = similar(A,size(A)[1:2]...) is redundant and can be omitted.
Without this unnecessary allocation, the results for the largest problem are:
#time rgb2gray_loop(rand(5000,5000,3));
#time rgb2gray_vec(rand(5000,5000,3));
elapsed time: 0.953746863 seconds (800000488 bytes allocated, 3.06% gc time)
elapsed time: 1.203013639 seconds (2200001200 bytes allocated, 7.28% gc time)
So the memory usage went down, but the speed did not noticeably improve.