I have functions which are called many times and require temporary arrays. Rather than array allocation happening every time the function is called, I would like the temporary to be statically allocated once.
How do I create a statically allocated array in Julia, with function scope?
Ok, let's assume that your function is called foo with an argument x and your array is just 100 hundred elements (each of which is a 64-bit value) with one dimension. Then you can create a scope around that function
let
global foo
let A = Array{Int64}(100)
function foo(x)
# do your tasks
end
end
A should be a let variable since it would overwrite any other global A.
You can wrap the temp array as a reference in a class:
type MyWrapper
thetmparray
thefunction::Function
function MyWrapper(outertmp::Array)
this = new(outertmp)
this.thefunction = function()
#use this.thetmparray or outertmp
end
return this
end
end
This away you can avoid global variables and (in the future) have a per-executor/thread/process/machine/etc temp array.
You can use either the let block or partial application (I prefer this approach for such a case):
function bind_array(A::Array)
function f(x)
A = A*x
end
end
Now you can bind a private array to every new "instance" of your f:
julia> f_x = bind_array(ones(1,2))
f (generic function with 1 method)
julia> display(f_x(2))
1x2 Array{Float64,2}:
2.0 2.0
julia> display(f_x(3))
1x2 Array{Float64,2}:
6.0 6.0
julia> f_y = bind_array(ones(3,2))
f (generic function with 1 method)
julia> display(f_y(2))
3x2 Array{Float64,2}:
2.0 2.0
2.0 2.0
2.0 2.0
julia> display(f_y(3))
3x2 Array{Float64,2}:
6.0 6.0
6.0 6.0
6.0 6.0
Related
I'm currently working on creating a subtype of AbstractArray in Julia, which allows you to store a vector in addition to an Array itself. You can think of it as the column "names", with element types as a subtype of AbstractFloat. Hence, it has some similarities to the NamedArray.jl package, but restricts to only assigning the columns with Floats (in case of matrices).
The struct that I've created so far (following the guide to create a subtype of AbstractArray) is defined as follows:
struct FooArray{T, N, AT, VT} <: AbstractArray{T, N}
data::AT
vec::VT
function FooArray(data::AbstractArray{T1, N}, vec::AbstractVector{T2}) where {T1 <: AbstractFloat, T2 <: AbstractFloat, N}
length(vec) == size(data, 2) || error("Inconsistent dimensions")
new{T1, N, typeof(data), typeof(vec)}(data, vec)
end
end
#inline Base.#propagate_inbounds Base.getindex(fooarr::FooArray, i::Int) = getindex(fooarr.data, i)
#inline Base.#propagate_inbounds Base.getindex(fooarr::FooArray, I::Vararg{Int, 2}) = getindex(fooarr.data, I...)
#inline Base.#propagate_inbounds Base.size(fooarr::FooArray) = size(fooarr.data)
Base.IndexStyle(::Type{<:FooArray}) = IndexLinear()
This already seems to be enough to create objects of type fooArray and do some simple math with it. However, I've observed that some essential functions such as matrix-vector multiplications seem to be imprecise. For example, the following should consistently return a vector of 0.0, but:
R = rand(100, 3)
S = FooArray(R, collect(1.0:3.0))
y = rand(100)
S'y - R'y
3-element Vector{Float64}:
-7.105427357601002e-15
0.0
3.552713678800501e-15
While the differences are very small, they can quickly add up over many different calculations, leading to significant errors.
Where do these differences come from?
A look at the calculations via macro #code_llvm reveals that appearently different matmul functions from LinearAlgebra are used (with other minor differences):
#code_llvm S'y
...
# C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\LinearAlgebra\src\matmul.jl:111 within `*'
...
#code_llvm S'y
...
# C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\LinearAlgebra\src\matmul.jl:106 within `*'
...
Redefining the adjoint and * functions on our FooArray object provides the expected, correct result:
import Base: *, adjoint, /
Base.adjoint(a::FooArray) = FooArray(a.data', zeros(size(a.data, 1)))
*(a::FooArray{T, 2, AT, VT} where {AT, VT}, b::AbstractVector{S}) where {T, S} = a.data * b
S'y - R'y
3-element Vector{Float64}:
0.0
0.0
0.0
However, this solution (which is also done in NamedArrays here) would require defining and maintaining all sorts of functions, not just the standard functions in base, adding more and more dependencies just because of this small error margin.
Is there any simpler way to get rid of this issue without redefining every operation and possibly many other functions from other packages?
I'm using Julia version 1.6.1 on Windows 64-bit system.
Yes, the implementation of matrix multiplication will vary depending upon your array type. The builtin Array will use BLAS, whereas your custom fooArray will use a generic implementation, and due to the non-associativity of floating point arithmetic, these different approaches will indeed yield different values — and note that they may be different from the ground truth, even for the builtin Arrays!
julia> using Random; Random.seed!(0); R = rand(100, 3); y = rand(100);
julia> R'y - Float64.(big.(R)'big.(y))
3-element Vector{Float64}:
-3.552713678800501e-15
0.0
0.0
You may be able to implement your custom array as a DenseArray, which will ensure that it uses the same (BLAS-enabled) codepath. You just need to implement a few more methods, most importantly strides and unsafe_convert:
julia> struct FooArray{T, N} <: DenseArray{T, N}
data::Array{T, N}
end
Base.getindex(fooarr::FooArray, i::Int) = fooarr.data[i]
Base.size(fooarr::FooArray) = size(fooarr.data)
Base.IndexStyle(::Type{<:FooArray}) = IndexLinear()
Base.strides(fooarr::FooArray) = strides(fooarr.data)
Base.unsafe_convert(P::Type{Ptr{T}}, fooarr::FooArray{T}) where {T} = Base.unsafe_convert(P, fooarr.data)
julia> R = rand(100, 3); S = FooArray(R); y = rand(100)
R'y - S'y
3-element Vector{Float64}:
0.0
0.0
0.0
julia> R = rand(100, 1000); S = FooArray(R); y = rand(100)
R'y == S'y
true
I want to move an array from my laptop (Julia 1.3.1) to my desktop PC (Julia 1.6.2).
I make an array in Julia 1.3.1 as follows.
using LinearAlgebra
H = ... #give a matrix H
eigen,vector = eigen(H)
Then, I'd like to move "vector" to Julia 1.6.2.
How do you do that?
The simplest way is by using DelimitedFiles:
julia> v = [1.0,2.0,3.0]
julia> using DelimitedFiles
julia> writedlm("f.txt", v)
julia> readdlm("f.txt")
3×1 Matrix{Float64}:
1.0
2.0
3.0
julia> vec(readdlm("f.txt"))
3-element Vector{Float64}:
1.0
2.0
3.0
Note that DelmitedFiles works with matrices so the last example shows what to do if you rather store a vector.
Edit following Bogumil's comment
When you have a Matrix of Complex numbers you need to provide the output type for readdlm:
julia> v = Complex.(rand(2,3), rand(2,3))
2×3 Matrix{ComplexF64}:
0.282157+0.540556im 0.757765+0.103518im 0.979935+0.212347im
0.557499+0.934859im 0.604032+0.338489im 0.431962+0.945946im
julia> writedlm("f.txt", v)
julia> readdlm("f.txt",'\t',Complex{Float64})
2×3 Matrix{ComplexF64}:
0.282157+0.540556im 0.757765+0.103518im 0.979935+0.212347im
0.557499+0.934859im 0.604032+0.338489im 0.431962+0.945946im
julia> readdlm("f.txt",'\t',Complex{Float64}) == v
true
Another way is to use a binary format. For long term in-between version serialization BSON (binary json) could be a good option:
julia> using BSON
julia> BSON.bson("v.bson", v = v)
julia> v2 = BSON.load("v.bson")[:v]
2×3 Matrix{ComplexF64}:
0.282157+0.540556im 0.757765+0.103518im 0.979935+0.212347im
0.557499+0.934859im 0.604032+0.338489im 0.431962+0.945946im
I am trying to return an array of different sized arrays in a Julia function.
In the function, the arrays will be initialized and, in a loop, they will have elements, that are other arrays, pushed to end of the array at each iteration. But I am getting the following error:
MethodError: no method matching push!(::Type{Array{Array{Float64,1},1}}, ::Array{Float64,1})
I am initializing an array of arrays:
x = Array{Array{Float64,1},1}
But when a push! other array, I get the error:
push!(x, y)
In python I would just append the new arrays to a list and return the list, how can I accomplish it in Julia?
Your statement:
julia> x = Array{Array{Float64,1},1}
Array{Array{Float64,1},1}
assigns to x name of the type.
In order to create an instance of this type add () after it:
julia> x = Array{Array{Float64,1},1}()
0-element Array{Array{Float64,1},1}
and now you can push! to it:
julia> push!(x, [2.5, 3.5])
1-element Array{Array{Float64,1},1}:
[2.5, 3.5]
Note that you could have initiated x with an empty vector accepting vectors of Float64 in the following way:
julia> x = Vector{Float64}[]
0-element Array{Array{Float64,1},1}
We use two features here:
Vector{Float64} is a shorthand for Array{Float64, 1}.
If you create an empty vector using [] syntax you can prepend a type of its elements in front of it just like I did in the example.
I have the following two structs. When I initialize the struct with a constructor, an array is created, but it is not what I expected.
using Distributions
mutable struct XYZ
x::Float64
y::Float64
z::Float64
end
mutable struct Coords
r::Vector{XYZ}
end
""" Make a structure called test that contains a vector of type XYZ
I want to have 10 XYZ structs in the "r" Vector """
Coords() = ( rand(Uniform(0,1.0),10,3) )
test = Coords()
I want to access test by going test.r.x[i], however Julia complains that type Tuple has no field r. What it does create is a 2 dimensional array of size 10x3 and I can call elements via test[i,j] but this is far from what I want. I want to have other arrays/variables in the composite with callable names...
I tried initializing this way as well
XYZ() = (rand(),rand(),rand())
Coords() = ([ XYZ() for i in 1:10 ])
test = Coords()
I still get the same warning. It seems like I have created a tuple rather than a composite type. How do I create a composite type that has arrays/vectors inside of other structs?
I am using Julia version 1.0.2
Your are never actually calling the inner constructor in Coords(). To achieve what you want:
XYZ() = XYZ(rand(), rand(), rand())
Coords() = Coords([XYZ() for _ in 1:10])
But I would recommend against providing a constructor that initializes a special random layout. Instead, you could properly overload rand for XYZ, which gives you an array-rand for free:
julia> import Random
julia> Random.rand(rng::Random.AbstractRNG, ::Random.SamplerType{XYZ}) = XYZ(rand(rng), rand(rng), rand(rng))
julia> rand(XYZ)
XYZ(0.7580070440261963, 0.15662533181464178, 0.7450476071687568)
julia> rand(XYZ, 10)
10-element Array{XYZ,1}:
XYZ(0.5984858021544213, 0.16235318543392796, 0.729919026616209)
XYZ(0.45516751074248374, 0.9667694185826785, 0.39798147467761247)
XYZ(0.7329129925610325, 0.7725520616259764, 0.42264014343531)
XYZ(0.10415869248789567, 0.4193162272272648, 0.3265074454289505)
XYZ(0.2286383169588948, 0.7119393337105202, 0.5166340562764509)
XYZ(0.23011692279595186, 0.35344093654843767, 0.9488399720160021)
XYZ(0.20464532213275577, 0.05761320898130973, 0.7694525743407523)
XYZ(0.3022492318001946, 0.9212313012991236, 0.819167833632835)
XYZ(0.6331585756351794, 0.9812979781832118, 0.3969247687412505)
XYZ(0.6049257667248391, 0.7155977104637223, 0.5294492917395452)
julia> Coords(rand(XYZ, 10))
Coords(XYZ[XYZ(0.633945, 0.882152, 0.750866), XYZ(0.496134, 0.241877, 0.188791), XYZ(0.267383, 0.552298, 0.613393), XYZ(0.569428, 0.503757, 0.120985), XYZ(0.822557, 0.982106, 0.37321), XYZ(0.250684, 0.701853, 0.509496), XYZ(0.886511, 0.83301, 0.381657), XYZ(0.628089, 0.00574949, 0.730268), XYZ(0.382186, 0.411701, 0.86016), XYZ(0.904469, 0.854098, 0.270464)])
Given a trivial function returning an array:
scala> def methodReturnsArray() = { Array(1.0, 2.0) }
methodReturnsArray: ()Array[Double]
We can go ahead and invoke the function:
scala> val myarr = methodReturnsArray
myarr: Array[Double] = Array(1.0, 2.0)
scala> myarr(0)
res21: Double = 1.0
However, it is not possible to use the apply ( / array index semantics) directly:
scala> methodReturnsArray(0)
<console>:53: error: too many arguments for method methodReturnsArray: ()Array[Double]
methodReturnsArray(0)
^
The request is to explain why that were not possible as given. Secondarily: is there some way to get an "inline" invocation. I.e. not requiring to separate out the steps: s
(a) invoke the method and
(b) access the specific element of the array (via apply())
on separate statements.
Aha! Seconds after posting this I arrived at the answer: need to include the parens on the function call: i.e methodReturnsArray () (0) :
scala> methodReturnsArray()(0)
res22: Double = 1.0