Does there exist a function similar to that of numpy's * operator for two arrays to multiply their elements in an element-wise manner, returning an array of the similar type?
For example:
#Lets define:
a = [0,1,2,3]
b = [1,2,3,4]
d = [[1,2] , [3,4], [5,6]]
e = [3,4,5]
#I want:
a * 2 == [2*0, 1*2, 2*2, 2*3]
a * b == [0*1, 1*2, 2*3, 3*4]
d * e == [[1*3, 2*3], [3*4, 4*4], [5*5, 6*5]]
d * d == [[1*1, 2*2], [3*3, 4*4], [5*5, 6*6]]
Note how * IS NOT regular matrix multiplication it is element-wise multiplication.
My current best solution is to write some c code, which does this, and import a compiled dll.
There must exist a better solution.
EDIT:
Using LabVIEW 2011 - Needs to be fast.
The first two multiplications can be done by using the 'multiply' primitive. Make sure the arrays in the second case are of the same length.
For the third multipllication you can use a for loop (with auto-indexing). This is needed because you need to instruct LabVIEW what the basic index is.
The last multiplication can (again) be done using the multiply primitive.
My result is different (opposite) from the previous posters. I generated a 4x1000 array of random numbers (magnitude 1000) which I multiplied by a 4x4 array of integers (1,2,3,4,...). I did this 100,000 times using the matrix multiplication VI and also using for loops to perform the operation on the arrays. I'm seeing times on the order of 0.328s for the matrix VIs and 0.051s for the for loops. Using a compiled DLL may be faster than Labview, but this does not seem to be true for the built-in functions.
This is certainly not what I expected, but it is consistent over many cycles. The VI is standard execution thread. All data types are set before the timed operations - no coercion takes place in the loops. The operations are performed separately, staged by a flat sequence structure, as is the time measurement. Parallelism is turned off.
Related
Going thought the Julia's performance tips I haven't found any suggestions regarding how to speed up a code with three dimensional arrays.
From my understanding d-element Array{Array{Float64,2},1} would perform best when d (the third dimension) is small. However, I am not sure whether this is the case when d is large.
Is there any tutorial on this topic for Julia?
Example 1a (d=50)
x = [zeros(100, 10) for d=1:50];
#time for d=1:50
x[d] = rand(100,10);
end
0.000100 seconds (50 allocations: 396.875 KB)
Example 1b (d=50)
y=zeros(100, 10, 50);
#time for d=1:50
y[:,:,d] = rand(100,10);
end
0.000257 seconds (200 allocations: 400.781 KB)
Example 2a (d=50000)
x = [zeros(100, 10) for d=1:50000];
#time for d=1:50000
x[d] = rand(100,10);
end
0.410813 seconds (99.49 k allocations: 388.328 MB, 81.88% gc time)
Example 2b (d=50000)
y=zeros(100, 10, 50000);
#time for d=1:50000
y[:,:,d] = rand(100,10);
end
0.185929 seconds (298.98 k allocations: 392.898 MB, 6.83% gc time)
From my understanding d-element Array{Array{Float64,2},1} would perform best when d (the third dimension) is small. However, I am not sure whether this is the case when d is large.
No, it's moreso how you use it. A = Array{Array{Float64,2},1} is an array of pointers to matrices. The value of an array is the pointer or the reference. Thus A[i] returns a reference, i.e. it's cheap. A2 = Array{Float64,3} is a contiguous array of floats. It's really just an indexing setup over a linear slab of memory (and has a linear index A2[i] which runs through the whole thing using that linear form).
The latter has some advantages because it is contiguous. There's no indirection, so looping over all of A2s values will be faster. A has to deference two pointers to get a value, so a simple 3D loop will be slower if you don't know to deference each internal matrix only once. Also, you can get views to the matrices via #view A2[:,:,1] etc., but you have to take note that A2[:,:,1] by itself will make a copy of the matrix. A[1] is natural a view because it returns the reference to the matirx, and if you want to copy you'd have to explicitly do copy(A[1]). Because A is just a linear array of pointers, push!ing a new matrix onto it is cheap since it's just increasing a relatively small array (and push! is automatically amortized) to add a new pointer on the end (this is why things like DifferentialEqautions.jl use arrays of arrays to build timeseries instead of the more traditional matrix).
So they are different tools with different advantages and disadvantages.
As for your timings, you're doing two different things. x[d] = rand(100,10) is creating a new matrix and adding its reference to x. y[:,:,d] = rand(100,10) is creating a new matrix and looping through the values of y to change the values of y. You can see why that's slower. But what you're leaving out is the allocation-free cases.
function f2()
y=zeros(100, 10, 50);
#time for i in eachindex(y)
y[i] = rand()
end
y
end
In the small case this matches the array creation. You can't naively do this on case one, but as I said, if you dereference the pointer for the matrix once you do really well:
function f()
x = [zeros(100, 10) for d=1:5000];
#time #inbounds for d=1:50
xd = x[d]
for i in eachindex(xd)
xd[i] = rand()
end
end
x
end
So arrays of arrays can be great data structures in the right cases. The library RecursiveArrayTools.jl was created to take better advantage of it. For example, A3 = VectorOfArrays(A) gives A3 the same indexing structure as A2 by lazily transforming A[i,j,k] to A[k][i,j]. However, it keeps the advantages of A, but will automatically make sure to broadcast in the correct way like f. Another tool like this is the ArrayPartition which allows for heterogeneous typing in a broadcast-performant way.
So yeah, it's not always the right tool, but these heterogeneous and recursive arrays are great tools when used correctly.
I am very new to Haskell (and functional programming in general) and I am trying to write a function called
"profileDistance m1 m2" that takes two matrices as parameters and needs to calculate the sum of the differences between each element in each matrix... I might have not explained that very well. Let me show it instead.
The matrices are on the form of: [[(Char,Int)]]
where each matrix might look something like this:
m1 = [[('A',1),('A',2)],
[('B',3),('B',4)],
[('C',5),('C',6)]]
m2 = [[('A',7),('A',8)],
[('B',9),('B',10)],
[('C',11),('C',12)]]
(Note: I wrote the numbers in order in this example but they can be ANY numbers in any order. The chars in each row in each matrix will however match like shown in the example.)
The result (in the case above) would look something like (psuedo code):
result = ((snd m1['A'][0])-(snd m2['A'][0]))+((snd m1['A'][1])-(snd m2['A'][1]))+((snd m1['B'][0])-(snd m2['B'][0]))+((snd m1['B'][1])-(snd m2['B'][1]))+((snd m1['C'][0])-(snd m2['C'][0]))+((snd m1['C'][1])-(snd m2['C'][1]))
This would be easy to do in any language that has for-loops and is non-functional but I have no idea how to do this in Haskell. I have a feeling that functions like map, fold or sum would help me here (admittedly I am not a 100% sure on how fold works). I hope there is an easy way to do this... please help.
Here a proposal:
solution m1 m2 = sum $ zipWith diffSnd flatM1 flatM2
where
diffSnd t1 t2 = snd t1 - snd t2
flatM1 = concat m1
flatM2 = concat m2
I wrote it so that it's easier to understand the building blocks.
The basic idea is to iterate simultaneously on our two lists of pairs using zipWith. Here its type:
zipWith :: (a -> b -> c) -> [a] -> [b] -> [c]
It means it takes a function with type a -> b -> c, a list of a's and a list of b's, and it returns a list of c's. In other words, zipWith takes case of the iteration, you just have to specify what you want to do with every item the iteration yields, that in your case will be a pair of pairs (one from the first matrix, another one from the second).
The function passed to zipWith takes the snd element from each pair, and computes the difference. Looking back at zipWith signature you can deduce it will return a list of numbers. So the last thing we need to do is summing them, using the function sum.
There's one last problem. We actually do not have two lists of pairs to be passed to zipWith!, but two matrices. We need to "flatten" them in a list, preserving the order of the elements. That's exactly what concat does, hence the calls to that function in the definitions of flatM1 and flatM2.
I suggest you look into the implementation of every function I mentioned to have a better grasp of how iteration is expressed by mean of recursion. HTH
Suppose that f(x,y) is a bivariate function as follows:
function [ f ] = f(x,y)
UN=(g)1.6*(1-acos(g)/pi)-0.8;
f= 1+UN(cos(0.5*pi*x+y));
end
How to improve execution time for function F(N) with the following code:
function [VAL] = F(N)
x=0:4/N:4;
y=0:2*pi/1000:2*pi;
VAL=zeros(N+1,3);
for i = 1:N+1
val = zeros(1,N+1);
for j = 1:N+1
val(j) = trapz(y,f(0,y).*f(x(i),y).*f(x(j),y))/2/pi;
end
val = fftshift(fft(val))/N;
l = (length(val)+1)/2;
VAL(i,:)= val(l-1:l+1);
end
VAL = fftshift(fft(VAL,[],1),1)/N;
L = (size(VAL,1)+1)/2;
VAL = VAL(L-1:L+1,:);
end
Note that N=2^p where p>10, so please consider the memory limitations while optimizing the code using ndgrid, arrayfun, etc.
FYI: The code intends to find the central 3-by-3 submatrix of the fftn of
fun=#(a,b) trapz(y,f(0,y).*f(a,y).*f(b,y))/2/pi;
where a,b are in [0,4]. The key idea is that we can save memory using the code above specially when N is very large. But the execution time is still an issue because of nested loops. See the figure below for N=2^2:
This is not a full answer, but some possibly helpful hints:
0) The trivial: Are you sure you need numerics? Can't you do the computation analytically?
1) Do not use function handles:
function [ f ] = f(x,y)
f= 1+1.6*(1-acos(cos(0.5*pi*x+y))/pi)-0.8
end
2) Simplify analytically: acos(cos(x)) is the same as abs(mod(x + pi, 2 * pi) - pi), which should compute slightly faster. Or, instead of sampling and then numerically integrating, first integrate analytically and sample the result.
3) The FFT is a very efficient algorithm to compute the full DFT, but you don't need the full DFT. Since you only want the central 3 x 3 coefficients, it might be more efficient to directly apply the DFT definition and evaluate the formula only for those coefficients that you want. That should be both fast and memory-efficient.
4) If you repeatedly do this computation, it might be helpful to precompute DFT coefficients. Here, dftmtx from the Signal Processing toolbox can assist.
5) To get rid of the loops, think about the problem not in the form of computation instructions, but a single matrix operation. If you consider your input N x N matrix as a vector with N² elements, and your output 3 x 3 matrix as a 9-element vector, then the whole operation you apply (numerical integration via trapz and DFT via fft) appears to be a simple linear transform, which it should be possible to express as an N² x 9 matrix.
I have accepted an answer to the question below, but It seemed I misunderstood how Arrays in haskell worked. I thought they were just beefed up lists. Keep that in mind when reading the question below.
I've found that monolithic arrays in haskell are quite inefficient when using them for larger arrays.
I haven't been able to find a non-monolithic implementation of arrays in haskell. What I need is O(1) time look up on a multidimensional array.
Is there an implementation of of arrays that supports this?
EDIT: I seem to have misunderstood the term monolithic. The problem is that it seems like the arrays in haskell treats an array like a list. I might be wrong though.
EDIT2: Short example of inefficient code:
fibArray n = a where
bnds = (0,n)
a = array bnds [ (i, f i) | i <- range bnds ]
f 0 = 0
f 1 = 1
f i = a!(i-1) + a!(i-2)
this is an array of length n+1 where the i'th field holds the i'th fibonacci number. But since arrays in haskell has O(n) time lookup, it takes O(n²) time to compute.
You're confusing linked lists in Haskell with arrays.
Linked lists are the data types that use the following syntax:
[1,2,3,5]
defined as:
data [a] = [] | a : [a]
These are classical recursive data types, supporting O(n) indexing and O(1) prepend.
If you're looking for multidimensional data with O(1) lookup, instead you should use a true array or matrix data structure. Good candidates are:
Repa - fast, parallel, multidimensional arrays -- (Tutorial)
Vector - An efficient implementation of Int-indexed arrays (both mutable and immutable), with a powerful loop optimisation framework . (Tutorial)
HMatrix - Purely functional interface to basic linear algebra and other numerical computations, internally implemented using GSL, BLAS and LAPACK.
Arrays have O(1) indexing. The problem is that each element is calculated lazily. So this is what happens when you run this in ghci:
*Main> :set +s
*Main> let t = 100000
(0.00 secs, 556576 bytes)
*Main> let a = fibArray t
Loading package array-0.4.0.0 ... linking ... done.
(0.01 secs, 1033640 bytes)
*Main> a!t -- result omitted
(1.51 secs, 570473504 bytes)
*Main> a!t -- result omitted
(0.17 secs, 17954296 bytes)
*Main>
Note that lookup is very fast, after it's already been looked up once. The array function creates an array of pointers to thunks that will eventually be calculated to produce a value. The first time you evaluate a value, you pay this cost. Here are a first few expansions of the thunk for evaluating a!t:
a!t -> a!(t-1)+a!(t-2)-> a!(t-2)+a!(t-3)+a!(t-2) -> a!(t-3)+a!(t-4)+a!(t-3)+a!(t-2)
It's not the cost of the calculations per se that's expensive, rather it's the need to create and traverse this very large thunk.
I tried strictifying the values in the list passed to array, but that seemed to result in an endless loop.
One common way around this is to use a mutable array, such as an STArray. The elements can be updated as they're available during the array creation, and the end result is frozen and returned. In the vector package, the create and constructN functions provide easy ways to do this.
-- constructN :: Unbox a => Int -> (Vector a -> a) -> Vector a
import qualified Data.Vector.Unboxed as V
import Data.Int
fibVec :: Int -> V.Vector Int64
fibVec n = V.constructN (n+1) c
where
c v | V.length v == 0 = 0
c v | V.length v == 1 = 1
c v | V.length v == 2 = 1
c v = let len = V.length v
in v V.! (len-1) + v V.! (len-2)
BUT, the fibVec function only works with unboxed vectors. Regular vectors (and arrays) aren't strict enough, leading back to the same problem you've already found. And unfortunately there isn't an Unboxed instance for Integer, so if you need unbounded integer types (this fibVec has already overflowed in this test) you're stuck with creating a mutable array in IO or ST to enable the necessary strictness.
Referring specifically to your fibArray example, try this and see if it speeds things up a bit:
-- gradually calculate m-th item in steps of k
-- to prevent STACK OVERFLOW , etc
gradualth m k arr
| m <= v = pre `seq` arr!m
where
pre = foldl1 (\a b-> a `seq` arr!b) [u,u+k..m]
(u,v) = bounds arr
For me, for let a=fibArray 50000, gradualth 50000 10 aran at 0.65 run time of just calling a!50000 right away.
I often find myself wanting to collapse an n-dimensional matrix across one dimension using a custom function, and can't figure out if there is a concise incantation I can use to do this.
For example, when parsing an image, I often want to do something like this. (Note! Illustrative example only. I know about rgb2gray for this specific case.)
img = imread('whatever.jpg');
s = size(img);
for i=1:s(1)
for j=1:s(2)
bw_img(i,j) = mean(img(i,j,:));
end
end
I would love to express this as something like:
bw = on(color, 3, #mean);
or
bw(:,:,1) = mean(color);
Is there a short way to do this?
EDIT: Apparently mean already does this; I want to be able to do this for any function I've written. E.g.,
...
filtered_img(i,j) = reddish_tint(img(i,j,:));
...
where
function out = reddish_tint(in)
out = in(1) * 0.5 + in(2) * 0.25 + in(3) * 0.25;
end
Many basic MATLAB functions, like MEAN, MAX, MIN, SUM, etc., are designed to operate across a specific dimension:
bw = mean(img,3); %# Mean across dimension 3
You can also take advantage of the fact that MATLAB arithmetic operators are designed to operate in an element-wise fashion on matrices. For example, the operation in your function reddish_tint can be applied to all pixels of your image with this single line:
filtered_img = 0.5.*img(:,:,1)+0.25.*img(:,:,2)+0.25.*img(:,:,3);
To handle a more general case where you want to apply a function to an arbitrary dimension of an N-dimensional matrix, you will probably want to write your function such that it accepts an additional input argument for which dimension to operate over (like the above-mentioned MATLAB functions do) and then uses some simple logic (i.e. if-else statements) and element-wise matrix operations to apply its computations to the proper dimension of the matrix.
Although I would not suggest using it, there is a quick-and-dirty solution, but it's rather ugly and computationally more expensive. You can use the function NUM2CELL to collect values along a dimension of your array into cells of a cell array, then apply your function to each cell using the function CELLFUN:
cellArray = num2cell(img,3); %# Collect values in dimension 3 into cells
filtered_img = cellfun(#reddish_tint,cellArray); %# Apply function to each cell
I wrote a helper function called 'vecfun' that might be useful for this, if it's what you're trying to achieve?
link
You could use BSXFUN for at least some of your tasks. It performs an element-wise operation among two arrays by expanding the size 1 - dimensions to match the size in the other array. The 'reddish tint' function would become
reddish_image = bsxfun(#times,img,cat(3,0.5,0.25,0.25));
filtered_img = sum(reddish_image,3);
All the above statement requires in order to work is that the third dimension of img has size 1 or 3. Number and size of the other dimensions can be chosen freely.
If you are consistently trying to apply a function to a vector comprised by the 3 dimension in a block of images, I recommend using a pair reshapes, for instance:
Img = rand(480,640,3);
sz = size(Img);
output = reshape(myFavoriteFunction(reshape(Img,[prod(sz(1:2)),sz(3)])'),sz);
This way you can swap in any function that operates on matrices along their first dimension.
edit.
The above code will crash if you input an image which has only one layer: The function below can fix it.
function o = nLayerImage2MatrixOfPixels(i)
%function o = nLayerImage2MatrixOfPixels(i)
s = size(i);
if(length(s) == 2)
s3 = 1;
else
s3 = s(3);
end
o = reshape(i,[s(1)*s(2),s(3)])';
Well, if you are only concerned with multiplying vectors together you could just use the dot product, like this:
bw(:,:,1)*[0.3;0.2;0.5]
taking care that the shapes of your vectors conform.