I wonder is it a good idea to allocate temporary arrays in let-block, which wraps some function?
Some toy example: instead of
function foo(x)
y = zeros(100)
for i in 1 : 100
y[i] = 2*x[i] - 1
end
do_something(y)
end
I will write something like:
let
const y = zeros(100) # otherwise foo will be type-unstable due to 'global' y
function foo(x)
for i in 1 : 100
y[i] = 2*x[i] - 1
end
do_something(y)
end
end
It can be easily checked via #benchmark macro that in the second case memory for y-array will be allocated only once, which significantly improves performance (in my not-toy case). I'm wondering is it a "julian-way" to do such things?
I will give you an answer for Julia 1.0. For earlier versions of Julia the situation would be a bit different.
Point 1. Your code with let will not run under Julia 1.0 as let creates local scope and in local scope you are not allowed to use const.
Point 2. It is fully OK to do something like this in global scope:
const y = zeros(100) # otherwise foo will be type-unstable due to 'global' y
function foo(x)
for i in 1 : 100
y[i] = 2*x[i] - 1
end
do_something(y)
end
and you will have good performance as Julia 1.0 knows that y has constant type and will optimize it. The consequence is that you will have y in global scope and foo in methods table (i.e. you can call foo using its name).
Point 3. You can also use let block like this:
const foo = let y = zeros(100)
function inner_foo(x)
for i in 1 : 100
y[i] = 2*x[i] - 1
end
do_something(y)
end
end
This time y is defined only in local scope and does not leak out to global scope. Also inner_foo is not defined in global scope therefore you have to assign the return value of let block to a variable foo that then can be used to make calls (I make it a const to improve performance if it gets used in some functions later on)
Point 4. Note, however, that this almost identical code will not be as good, as Julia 1.0 has problems with type inference of variable y (hopefully this will be fixed in the future)
const foo = let
y = zeros(100)
function inner_foo(x)
for i in 1 : 100
y[i] = 2*x[i] - 1
end
do_something(y)
end
end
In summary: the decision if you use let block depends mostly on what you have to be visible in global scope (as what is defined in let is not visible in global scope) and if you use let block it is best to define variables you want to use as a part of let block definition.
Related
I have been banging my head against a wall trying to use static arrays in julia.
https://github.com/JuliaArrays/StaticArrays.jl
They are fast but updating them is a pain. This is no surprise, they are meant to be immutable!
But it is continuously recommended to me that I use static arrays even though I have to update them. In my case, the static arrays are small, just length 3, and i have a vector of them, but I only update 1 length three SVector at a time.
Option 1
There is a really neat package called Setfield that allows you to do inplace updates of SVectors in Julia.
https://github.com/jw3126/Setfield.jl
The catch... it updates the local copy. So if you are in a nested function, it updates the local copy. So it comes with some book keeping since you have to inplace update the local copy and then return that copy and update the actual array of interest. You can't pass in your desired array and update it in place, at least, not that I can figure out! Now, I do not mind bookeeping, but I feel like updating a local copy, then returning the value, updating another local copy, and then returning the values and finally updating the actual array must come with a speed penalty. I could be wrong.
Option 2
It bugs me that in order to do an update a static array I must
exampleSVector::SVector{3,Float64} <-- just to make clear its type and size
exampleSVector = [value1, value2, value3]
This will update the desired array even if it is inside a function, which is nice and the goal, but if you do this inside a function it creates a temporary array. And this kills me because my function is in a loop that gets called 4+ million times, so this creates a ton of allocations and slows things down.
How do I update an SVector for the Option 2 scenario without creating a temporary array?
For the Option 1 scenario, can I update the actual array of interest rather than the local copy?
If this requires a simple example code, please say so in the comments, and I will make one. My thinking is that it is answerable without one, but I will make one if it is needed.
EDIT:
MCVE code - Option 1 works, option 2 does not.
using Setfield
using StaticArrays
struct Keep
dreaming::Vector{SVector{3,Float64}}
end
function INNER!(vec::SVector{3,Float64},pre::SVector{3,Float64})
# pretend series of calculations
for i = 1:3 # illustrate use of Setfield (used in real code for this)
pre = #set pre[i] = rand() * i * 1000
end
# more pretend calculations
x = 25.0 # assume more calculations equals x
################## OPTION 1 ########################
vec = #set vec = x * [ pre[1], pre[2], pre[3] ] # UNCOMMENT FOR FOR OPTION 1
return vec # UNCOMMENT FOR FOR OPTION 1
################## OPTION 2 ########################
#vec = x * [ pre[1], pre[2], pre[3] ] # UNCOMMENT FOR FOR OPTION 2
#nothing # UNCOMMENT FOR FOR OPTION 2
end
function OUTER!(always::Keep)
preAllocate = SVector{3}(0.0,0.0,0.0)
for i=1:length(always.dreaming)
always.dreaming[i] = INNER!(always.dreaming[i], preAllocate) # UNCOMMENT FOR FOR OPTION 1
#INNER!(always.dreaming[i], preAllocate) # UNCOMMENT FOR FOR OPTION 2
end
end
code = Keep([zero(SVector{3}) for i=1:5])
OUTER!(code)
println(code.dreaming)
I hope that I have understood your question correctly. It's a bit hard with a MWE like this, that does a lot of things that are mostly redundant and a bit confusing.
There seems to be two alternative interpretations here: Either you really need to update ('mutate') an SVector, but your MWE fails to demonstrate why. Or, you have convinced yourself that you need to mutate, but you actually don't.
I have decided to focus on alternative 2: You don't really need to 'mutate'. Rewriting your code from that point of view simplifies it greatly.
I couldn't find any reason for you to mutate any static vectors here, so I just removed that. The behaviour of the INNER! function with the inputs was very confusing. You provide two inputs but don't use either of them, so I removed those inputs.
function inner()
pre = #SVector [rand() * 1000i for i in 1:3]
x = 25
return pre .* x
end
function outer!(always::Keep)
always.dreaming .= inner.() # notice the dot in inner.()
end
code = Keep([zero(SVector{3}) for i in 1:5])
outer!(code)
display(code.dreaming)
This runs fast and with zero allocations. In general with StaticArrays, don't try to mutate things, just create new instances.
Even though it's not clear from your MWE, there may be some legitimate reason why you may want to 'mutate' an SVector. In that case you can use the setindex method of StaticArrays, you don't need Setfield.jl:
julia> v = rand(SVector{3})
3-element SArray{Tuple{3},Float64,1,3}:
0.4730258499237898
0.23658547518737905
0.9140206579322541
julia> v = setindex(v, -3.1, 2)
3-element SArray{Tuple{3},Float64,1,3}:
0.4730258499237898
-3.1
0.9140206579322541
To clarify: setindex (without a !) does not mutate its input, but creates a new instance with one index value changed.
If you really do need to 'mutate', perhaps you can make a new MWE that shows this. I would recommend that you try to simplify it a bit, because it is quite confusing now. For example, the inclusion of the type Keep seems entirely unnecessary and distracting. Just make a Vector of SVectors and show what you want to do with that.
Edit: Here's an attempt based on the comments below. As far as I understand it now, the question is about modifying a vector of SVectors. You cannot really mutate the SVectors, but you can replace them using a convenient syntax, setindex, where you can keep some of the elements and change some of the others:
oldvec = [zero(SVector{3}) for _ in 1:5]
replacevec = [rand(SVector{3}) for _ in 1:5]
Now we replace the second element of each element of oldvec with the corresponding one in replacevec. First a one-liner:
oldvec .= setindex.(oldvec, getindex.(replacevec, 2), 2)
Then an even faster one with a loop:
for i in eachindex(oldvec, replacevec)
#inbounds oldvec[i] = setindex(oldvec[i], replacevec[i][2], 2)
end
There are two types of static arrays - mutable (starting with M in type name) and immutable ones (starting with S) - just use the mutable ones! have a look at the example below:
julia> mut = MVector{3,Int64}(1:3);
julia> mut[1]=55
55
julia> mut
3-element MArray{Tuple{3},Int64,1,3}:
55
2
3
julia> immut = SVector{3,Int64}(1:3);
julia> inmut[1]=55
ERROR: setindex!(::SArray{Tuple{3},Int64,1,3}, value, ::Int) is not defined.
Let us see some simple benchmark (ordinary array, vs mutable static vs immutable static):
using BenchmarkTools
julia> ord = [1,2,3];
julia> #btime $ord.*$ord;
39.680 ns (1 allocation: 112 bytes)
3-element Array{Int64,1}:
1
4
9
julia> #btime $mut.*$mut
8.533 ns (1 allocation: 32 bytes)
3-element MArray{Tuple{3},Int64,1,3}:
3025
4
9
julia> #btime $immut.*$immut
2.133 ns (0 allocations: 0 bytes)
3-element SArray{Tuple{3},Int64,1,3}:
1
4
9
I am using Lua v5.2.2 within a C application (embedded environment/MCU).
I need to expose some "parameters" in Lua, that for reading and writing you need to directly access the hardware (thus a C call is needed). I am looking however for other means to implement this than using plain old getters and setters.
I am mostly exploring the meta-programming power of Lua, but also I believe I can create a simpler interface for the user.
What I want to achieve is behaviour like the following:
my_param = createParameter{name="hw_param1", type="number", min=0, max=100}
my_param = 5
result = my_param + 3
On the first line a new parameter is created. This is a call towards a C function. Userdata is pushed to stack with a properly initialized struct. The hardware is also initialized as needed. A new table is returned.
On the second line an assignment is done to the parameter object. I want this to call a C function with a single argument (that of the assignment), so the value can be stored to the hardware registers.
On the third line the parameter is read. I again need a call towards a C function that will get the value of the parameter from the hardware registers, and that will return the result.
Note that the actual value of this parameter may change outside the scope of Lua, so reading the value once during initialization is not correct. The C function must be called each time to get the actual value. Similarly writing to the value must cause an immediate write to the hardware.
How can I accomplish this? Specifically can I alter the metatable of the parameter to achieve lines 2 and 3? (I am aware of how to implement line 1).
Also is it necessary to return a table from the constructor? May I, for example, return a primitive Lua type (e.g. a number) that will behave like above?
Yes, you can modify the metatable metamethods.
Line 2 would completely change the variable's value that it holds.
However, if you were to set a field in the parameter object like: my_param.x = n, the __newindex metamethod would get invoked; which you could overwrite the metamethod to behave as you would like. In your case you would make it set the parameter's field and update it with a C function call.
Regarding line 3, same principle applies, instead this time you would just use the __add metamethod, and manipulate the object when __add is invoked.
http://lua-users.org/wiki/MetamethodsTutorial
This isn't exactly what you're asking for, but it's close:
function createParameter(t)
param = {}
param.data = t
backingTable = {}
metatable = {}
function metatable.__index(t, k)
-- You can intercept the value here if you
-- want and pass it on to your C fuction.
return backingTable[k]
end
function metatable.__newindex(t, k, v)
-- You can intercept the value here if you
-- want and pass it on to your C fuction.
backingTable[k] = v
end
setmetatable(param, metatable)
return param
end
--------------------------------------------------------
my_param = createParameter{name="hw_param1", type="number", min=0, max=100}
my_param.value = 5
result = my_param.value + 3
print(result) -- prints 8
print(my_param.data.name) -- prints hw_param1
You might be able to do something tricky by assigning a metatable to the global table _G, but I think that would be kind of tricky to get set up right and could lead to unexpected outcomes.
Edit:
If you really hate having to have a level of indirection, and you really want to be able to set it directly, here's how you can do it by setting the global table.
globalMetatable = {}
globalParamNames = {}
globalParams = {}
function globalMetatable.__index(t, k)
if globalParamNames[k] then
-- You can intercept the value here if you
-- want and pass it on to your C fuction.
print("Read from param " .. k)
return globalParams[k]
else
rawget(_G, k)
end
end
function globalMetatable.__newindex(t, k, v)
if globalParamNames[k] then
-- You can intercept the value here if you
-- want and pass it on to your C fuction.
print("Wrote to param " .. k)
globalParams[k] = v
else
rawset(_G, k, v)
end
end
setmetatable(_G, globalMetatable)
function createParameter(t)
globalParamNames[t.varname] = true
end
--------------------------------------------------------
createParameter{varname="my_param", name="hw_param1", type="number", min=0, max=100}
my_param = 5
result = my_param + 3
print(result) -- prints 8
print(my_param) -- prints 5
I have a dictionary that has 3 key, each key has arbitrary 2d arrays. And I want to assign each of these values(2d arrays) to newly created arrays. So I wrote this :
levels = (:easy, :medium, :hard)
easy = []; medium = []; hard = [];
curriculum = Dict((k=>rand(3,3) for k in levels)...); # dictionary with 3 keys - each key is a symbol
for level in levels
eval(level) = curriculum[level]
end
There is no problem during execution but when I check the easy, medium or hard arrays after, there are no matrices inside.
What is the proper way of doing this?
B.R.
(edited)
What I understand is that during the loop iteration eval(level) behaves like local binding. Thus, at the end of the iteration it does not remember anything. But why? And how to convert to global again ?
There is no problem during execution but when I check the easy, medium or hard arrays after, there are no matrices inside.
What I understand is that during the loop iteration eval(level) behaves like local binding. Thus, at the end of the iteration it does not remember anything. But why? And how to convert to global again ?
The problem is that you are defining and then redefining an eval function in each iteration inside the for loop, instead of doing assignments as you believe.
easy = []; medium = []; hard = []
1.- There is no need to predefine this variables.
eval(level) = curriculum[level]
2.- Warning: This is being interpreted as a short style function definition!
julia> eval(level) = curriculum[level]
eval (generic function with 1 method)
julia> levels = :easy, :medium, :hard
(:easy, :medium, :hard)
julia> curriculum = Dict((k => rand(3, 3) for k in levels)...)
Dict{Symbol,Array{Float64,2}} with 3 entries:
:medium => [0.230877 0.244128 0.205712; 0.649405 0.887568 0.957849; 0.245076 0.476088 0.160561]
:hard => [0.0424943 0.157261 0.798607; 0.590535 0.78125 0.704322; 0.555827 0.675355 0.74026]
:easy => [0.715595 0.914093 0.517973; 0.750787 0.489784 0.491666; 0.1966 0.133281 0.249144]
julia> for level in levels
#eval $level = curriculum[$(Meta.quot(level))]
end
julia> easy
3×3 Array{Float64,2}:
0.715595 0.914093 0.517973
0.750787 0.489784 0.491666
0.1966 0.133281 0.249144
3.- Inside the for loop, in the first iteration, $level is interpreted as the easy variable identifier, while $(Meta.quot(level)) is interpreted as the symbol :easy, for which the extra level of quoting is needed.
julia> Meta.quot(:foo)
:(:foo)
julia> Expr(:quote, :foo)
:(:foo)
Edit there is a better way as pointed out in the accepted answer and comment by Ismael. Namely that:
global level = level is not necessary, see my answer, the issue is
that if you want to interpolate a symbol into an expression as a
symbol instead of as an identifier, then you need to wrap the symbol
into another quote expression.
Warning: this is (almost certainly) not the best way of setting out to accomplish whatever you want to do. Metaprogramming in julia is super powerful and useful but you should very rarely need to use eval. You can read more about it here. If you ask another question with the details of your end-goal you might get a better answer :)
Nonetheless:
If you want to do the assignment this way you'll have to create an Expression Expr that assigns the matrix object to the variable with the name you want. But because eval always executes the statement in the global scope you'll need to make the variable you want to use available in the global scope.
for level in levels
global level = level
eval(:($level = curriculum[level]))
end
or using the #eval macro:
for level in levels
global level = level
#eval $level = curriculum[level]
end
I have just started using Julia. I am trying to use eval (in Julia) in order to define a set of variables in a function. Let's say I want to set v1 equal to 2:
function fun_test(varargs...)
v1 = 0;
if length(varargs) > 0
j = collect(linspace(1,length(varargs)-1,length(varargs)/2));
for i in j
expr_vargs = parse("$(varargs[i]) = $(varargs[i+1]);");
eval(expr_vargs);
end
end
println(v1)
end
Calling the function as:
fun_test("v1", "2");
It doesn't work, since println returns 0 (the initial value of v1). However, if I run an analogous eval call in the Julia's terminal, then it works.
Could you please clarify why it doesn't work and how to fix it?
eval runs in toplevel scope, not in function scope. It is not possible to dynamically update bindings in function scope. Without knowing your precise use case, I suspect there is a way to do things without dynamic rebinding. In particular, v1, v2, etc. is probably best made into an array, V.
Nevertheless, if you really must, you can always define v1 as a global variable in a submodule:
module FunTest
v1 = 0
function fun_test(varargs...)
if length(varargs) > 0
j = collect(linspace(1,length(varargs)-1,length(varargs)/2));
for i in j
#eval $(varargs[i]) = $(varargs[i+1])
end
end
println(v1)
end
export fun_test
end
using .FunTest
fun_test(:v1, 2) # result: 2
(I have also modified your code to avoid parseing strings, which is best done through expression interpolation.)
Consider the following code snippet
for i = 1:100
Yi= x(i:i + 3); % i in Yi is not an index but subscript,
% x is some array having sufficient values
i = i + 3
end
Basically I want that each time the for loop runs the subscript changes from 1 to 2, 3, ..., 100. SO in effect after 100 iterations I will be having 100 arrays, starting with Y1 to Y100.
What could be the simplest way to implement this in MATLAB?
UPDATE
This is to be run 15 times
Y1 = 64;
fft_x = 2 * abs(Y1(5));
For simplicity I have taken constant inputs.
Now I am trying to use cell based on Marc's answer:
Y1 = cell(15,1);
fft_x = cell(15,1);
for i = 1:15
Y1{i,1} = 64;
fft_x{i,1} = 2 * abs(Y1(5));
end
I think I need to do some changes in abs(). Please suggest.
It is impossible to make variably-named variables in matlab. The common solution is to use a cell array for Y:
Y=cell(100,1);
for i =1:100
Y{i,1}= x(i:i+3);
i=i+3;
end
Note that the line i=i+3 inside the for-loop has no effect. You can just remove it.
Y=cell(100,1);
for i =1:100
Y{i,1}= x(i:i+3);
end
It is possible to make variably-named variables in matlab. If you really want this do something like this:
for i = 1:4:100
eval(['Y', num2str((i+3)/4), '=x(i:i+3);']);
end
How you organize your indexing depends on what you plan to do with x of course...
Yes, you can dynamically name variables. However, it's almost never a good idea and there are much better/safer/faster alternatives, e.g. cell arrays as demonstrated by #Marc Claesen.
Look at the assignin function (and the related eval). You could do what asked for with:
for i = 1:100
assignin('caller',['Y' int2str(i)],rand(1,i))
end
Another related function is genvarname. Don't use these unless you really need them.