I'm using Clojure and I need to run a small simulation. I have a vector of length n (n is usually between 10 and 100) that holds values. On each simulation round (maybe 1000 rounds together), one of the values in the vector is updated randomly. I guess I could do this by using an Java array and calling the aset method, but this would break the functional programming/immutability idiom.
Is there a more functional way to do this, or should I just go with the Java array?
(defn run-sim [arr num-iters update-fn]
(if (zero? num-iters)
arr
(let [i (rand-int (count arr))
x (update-fn)]
(println "setting arr[" i "] to" x)
(recur (assoc arr i x) (dec num-iters) update-fn))))
user> (run-sim [1 2 3 4 5 6 7 8 9 10] 10 #(rand-int 1000))
setting arr[ 8 ] to 167
setting arr[ 4 ] to 977
setting arr[ 5 ] to 810
setting arr[ 5 ] to 165
setting arr[ 3 ] to 486
setting arr[ 1 ] to 382
setting arr[ 4 ] to 792
setting arr[ 8 ] to 478
setting arr[ 4 ] to 144
setting arr[ 7 ] to 416
[1 382 3 486 144 165 7 416 478 10]
There's no shame in using a Java array if you need it though. Especially if you need it to go fast. Limit the array-mutation to the inside of your function (clone the input array and work on that maybe) and no one will be the wiser.
Adding to Brian's answer: If you need more speed, you can also resort to transients.
(defn run-sim
[vektor num-iters update-fn]
(loop [vektor (transient vektor)
num-iters (int num-iters)]
(if (zero? num-iters)
(persistent! vektor)
(let [i (rand-int (count vektor))
x (update-fn)]
(recur (assoc! vektor i x) (dec num-iters))))))
Lets first define a function which updates a random index in a vector with a new value. Note that the original vector is not changed, instead a new vector (with the updated value) is returned:
(defn f [xs]
(let [r (java.util.Random.)
i (.nextInt r (count xs))
b (.nextBoolean r)]
(assoc xs i ((if b inc dec) (xs i)))))
This function chooses an index and then it either increases or decreases the value at that index by 1. You must of course change this function to your needs.
Then it is a simple matter to compose this function with itself as many times you want to run the simulation:
user=> ((apply comp (repeat 1000 f)) [0 0 0 0 0 0 0])
[7 -4 7 6 10 0 -6]
It's not that Clojure won't let you change values, it's just a little more cumbersome.
(def vec-ref (ref my-vector))
(dosync (set! vec-ref (assoc my-vector index value))
to look at values in the changed vector, use #vec-ref.
Could be off in details - I'm not near a REPL, unfortunately. But it should get you started.
Related
I'm attempting to index an array based on conditions in the array itself and corresponding locations in other arrays. I've produced a MWE but this will be hopefully used in a much larger example, within a loop to automate across scenarios, dare I say... using parallelisation?
MWE:
# create 3d array
a, b, c = [8;8;6;5;5;6], [8;8;7;6;6;6], [8;2;7;7;6;6]
d = transpose(cat(a,b,c, dims = 2))
e, f, g = [3;2;5;1;4;1], [4;3;1;1;1;2], [5;1;2;1;2;3]
h = transpose(cat(e,f,g, dims = 2))
wrkarr = cat(d,h,dims = 3)
# create temp array for indexes
temp = wrkarr[size(wrkarr,1), :, 1]
# calculate indexes
temp[(wrkarr[size(wrkarr,1),:,1] .== 8) .& (wrkarr[size(wrkarr,1),:,2]) .>= 4] .= 22
In my case it doesn't change anything in the temp array, when I would expect the first element to be changed from 8 to 22. Both of the individual conditional tests produce a vector [1,0,0,0,0,0] so why won't the .& test produce the same? Thx. J
First, and as an aside, please note that you can use arr[end,i] to refer to the element arr with the last index in the 1st axis, and index i in the second axis.
Using this notation, your condition can be rewritten as:
julia> (wrkarr[end,:,1] .== 8) .& (wrkarr[end,:,2]) .>=4
6-element BitArray{1}:
0
0
0
0
0
0
It might be a bit easier to see here that there is a parenthesis issue. I think you actually wanted to write:
julia> (wrkarr[end,:,1] .== 8) .& (wrkarr[end,:,2] .>=4)
6-element BitArray{1}:
1
0
0
0
0
0
Making this change does what (I think) you want. (Also note that I added #views below in order to avoid allocations and speed things up a little)
julia> idx = (#view(wrkarr[end,:,1]) .== 8) .& (#view(wrkarr[end,:,2]) .>=4);
julia> temp[idx] .= 22
1-element view(::Array{Int64,1}, [1]) with eltype Int64:
22
julia> temp
6-element Array{Int64,1}:
22
2
7
7
6
6
EDIT: as mentioned in comments, other solutions could be considered:
using findall to generate a vector of indices matching the condition
using a simple for loop
Here is a benchmark of all three solutions, in order to see how they compare in terms of readability and performance.
TLDR: the for loop seems to be much more efficient in this case, and allocates less.
# create 3d array
d = [8 8 6 5 5 6;
8 8 7 6 6 6;
8 2 7 7 6 6]
h = [3 2 5 1 4 1;
4 3 1 1 1 2;
5 1 2 1 2 3]
wrkarr = cat(d,h,dims = 3)
using BenchmarkTools
Option 1 logical indexing
julia> function version1!(temp, wrkarr)
idx = (#view(wrkarr[end,:,1]) .== 8) .& (#view(wrkarr[end,:,2]) .>= 4)
temp[idx] .= 22
end
version1! (generic function with 1 method)
julia> temp = wrkarr[end, :, 1]; #btime version1!($temp, $wrkarr); temp
182.646 ns (3 allocations: 224 bytes)
6-element Array{Int64,1}:
22
2
7
7
6
6
Option 2 vector of indices
julia> function version2!(temp, wrkarr)
idx = findall(i -> (wrkarr[end,i,1] == 8) & (wrkarr[end,i,2] >=4), axes(wrkarr,2))
temp[idx] .= 22
end
version2! (generic function with 1 method)
julia> temp = wrkarr[end, :, 1]; #btime version2!($temp, $wrkarr); temp
134.395 ns (3 allocations: 208 bytes)
6-element Array{Int64,1}:
22
2
7
7
6
6
Option 3 for loop
julia> function version3!(temp, wrkarr)
#inbounds for i in axes(wrkarr, 2)
if wrkarr[end, i, 1] == 8 && wrkarr[end, i, 2] >= 4
temp[i] = 22
end
end
end
version3! (generic function with 1 method)
julia> temp = wrkarr[end, :, 1]; #btime version3!($temp, $wrkarr); temp
21.820 ns (0 allocations: 0 bytes)
6-element Array{Int64,1}:
22
2
7
7
6
6
Based on all the clever stuff given to me above I pushed it a bit further and this solution appears to work and dispenses with the need for an index array, simply does the substitution right back into the original array. Is there anything wrong with that? Looking at #François Févotte's answer the loop function is the fastest, and this could be very important when scaled up to proper size. Now for my next challenge I want to use the same function to loop through sets of numbers to replace the 8, 4, 22, for example for the same wrkarr:
== 8 &>= 4 -> 22
== 7 &>= 2 -> 8
== 6 &>= 3 -> 7
Any takers or suggestions? When I get that to work I want to see if it can be parallelized across these individual sets of numbers, i.e. each substitution can be done independently, not dependent on any other. Thx a bunch!
function version4(wrkarr)
#inbounds for i in axes(wrkarr, 2)
if wrkarr[end, i, 1] == 8 && wrkarr[end, i, 2] >= 4
wrkarr[end,i, 1] = 22
end
end
end
wrkarr[end, :, 1]; #btime version4($wrkarr); wrkarr
2.237 ns (0 allocations: 0 bytes)
3×6×2 Array{Int64,3}:
[:, :, 1] =
8 8 6 5 5 6
8 8 7 6 6 6
22 2 7 7 6 6
[:, :, 2] =
3 2 5 1 4 1
4 3 1 1 1 2
5 1 2 1 2 3
In Octave, I can do
octave:1> A = [1 2; 3 4]
A =
1 2
3 4
octave:2> A(A>1) -= 1
A =
1 1
2 3
but in Julia, the equivalent syntax does not work.
julia> A = [1 2; 3 4]
2x2 Array{Int64,2}:
1 2
3 4
julia> A[A>1] -= 1
ERROR: `isless` has no method matching isless(::Int64, ::Array{Int64,2})
in > at operators.jl:33
How do you conditionally assign values to certain array or matrix elements in Julia?
Your problem isn't with the assignment, per se, it's that A > 1 itself doesn't work. You can use the elementwise A .> 1 instead:
julia> A = [1 2; 3 4];
julia> A .> 1
2×2 BitArray{2}:
false true
true true
julia> A[A .> 1] .-= 1000;
julia> A
2×2 Array{Int64,2}:
1 -998
-997 -996
Update:
Note that in modern Julia (>= 0.7), we need to use . to say that we want to broadcast the action (here, subtracting by the scalar 1000) to match the size of the filtered target on the left. (At the time this question was originally asked, we needed the dot in A .> 1 but not in .-=.)
In Julia v1.0 you can use the replace! function instead of logical indexing, with considerable speedups:
julia> B = rand(0:20, 8, 2);
julia> #btime (A[A .> 10] .= 10) setup=(A=copy($B))
595.784 ns (11 allocations: 4.61 KiB)
julia> #btime replace!(x -> x>10 ? 10 : x, A) setup=(A=copy($B))
13.530 ns ns (0 allocations: 0 bytes)
For larger matrices, the difference hovers around 10x speedup.
The reason for the speedup is that the logical indexing solution relies on creating an intermediate array, while replace! avoids this.
A slightly terser way of writing it is
replace!(x -> min(x, 10), A)
There doesn't seem to be any speedup using min, though.
And here's another solution that is almost as fast:
A .= min.(A, 10)
and that also avoids allocations.
To make it work in Julia 1.0 one need to change = to .=. In other words:
julia> a = [1 2 3 4]
julia> a[a .> 1] .= 1
julia> a
1×4 Array{Int64,2}:
1 1 1 1
Otherwise you will get something like
ERROR: MethodError: no method matching setindex_shape_check(::Int64, ::Int64)
Is there a way to overwrite [] to have complement of range in array?
julia> a=[1:8...]
8-element Array{Int64,1}:
1
2
3
4
5
6
7
8
julia> a[-1] == a[2:8]
julia> a[-(1:3)] == a[4:8]
julia> a[-end] == a[1:7]
I haven't looked into the internals of indexing before, but at a first glance, the following might work without breaking too much:
immutable Not{T}
idx::T
end
if :to_indices in names(Base)
# 0.6
import Base: to_indices, uncolon, tail, _maybetail
#inline to_indices(A, inds, I::Tuple{Not, Vararg{Any}}) =
(setdiff(uncolon(inds, (:, tail(I)...)), I[1].idx), to_indices(A, _maybetail(inds), tail(I))...)
else
# 0.5
import Base: getindex, _getindex
not_index(a::AbstractArray, I, i::Int) = I
not_index(a::AbstractArray, I::Not, i::Int) = setdiff(indices(a, i), I.idx)
getindex(a::AbstractArray, I::Not) = getindex(a, setdiff(linearindices(a), I.idx))
_getindex(::Base.LinearIndexing, a::AbstractArray, I::Vararg{Union{Real, AbstractArray, Colon, Not}}) =
Base._getindex(Base.linearindexing(a), a, (not_index(a, idx, i) for (i,idx) in enumerate(I))...)
end
For example:
julia> a = reshape(1:9, (3, 3))
3×3 Base.ReshapedArray{Int64,2,UnitRange{Int64},Tuple{}}:
1 4 7
2 5 8
3 6 9
julia> a[Not(2:8)]
2-element Array{Int64,1}:
1
9
julia> a[Not(1:2), :]
1×3 Array{Int64,2}:
3 6 9
julia> a[Not(end), end]
2-element Array{Int64,1}:
7
8
I didn't care for performance and also did no extensive testing, so things can certainly be improved.
Edit:
I replaced the code for 0.6 with Matt B. version from his github comment linked in the comments.
Thanks to his great design of the array indexing implementation for 0.6, only a single function needs to be extended to get complement indexing for getindex, setindex and view, e.g.,
julia> view(a, Not(2:8))
2-element SubArray{Int64,1,UnitRange{Int64},Tuple{Array{Int64,1}},false}:
1
9
# collect because ranges are immutable
julia> b = collect(a); b[Not(2), Not(2)] = 10; b
3×3 Array{Int64,2}:
10 4 10
2 5 8
10 6 10
Directly overwriting [](i.e. getindex) is prone to break many indexing-related things in Base, but we can write an array wrapper to work around it. We only need to define the following three methods to get your specific test cases passed:
immutable ComplementVector{T} <: AbstractArray{T,1}
data::Vector{T}
end
Base.size(A:: ComplementVector) = size(A.data)
Base.getindex(A:: ComplementVector, i::Integer) = i > 0 ? A.data[i] : A.data[setdiff(1:end, (-i))]
Base.getindex(A:: ComplementVector, I::StepRange) = all(x->x>0, I) ? A.data[I] : A.data[setdiff(1:end, -I)]
julia> a = ComplementVector([1:8...])
julia> a[-1] == a[2:8]
true
julia> a[-(1:3)] == a[4:8]
true
julia> a[-end] == a[1:7]
true
If you would like to extend ComplementVector further more, please read the doc about Interfaces.
Update:
For safety sake, we'd better not extend AbstractArray as #Fengyang Wang suggested in the comment blow:
immutable ComplementVector{T}
data::Vector{T}
end
Base.endof(A::ComplementVector) = length(A.data)
Base.getindex(A::ComplementVector, i::Integer) = i > 0 ? A.data[i] : A.data[setdiff(1:end, (-i))]
Base.getindex(A::ComplementVector, I::OrdinalRange) = all(x->x>0, I) ? A.data[I] : A.data[setdiff(1:end, -I)]
I need to reset the 'lower triangle' of a 3 dimentional matrix.
This means, that if the original matrix is:
C(:,:,1) = [1 2 3 ; 2 4 6 ; 3 6 9]
C(:,:,2) = [2 4 6 ; 4 8 12 ; 6 12 18]
C(:,:,3) = [3 6 9 ; 6 12 18 ; 9 18 27]
Then the resulting matrix should be:
C(:,:,1) = [1 2 3 ; 2 4 6 ; 3 6 9]
C(:,:,2) = [0 0 0 ; 4 8 12 ; 6 12 18]
C(:,:,3) = [0 0 0 ; 0 0 0 ; 9 18 27]
Any idea how such a thing csn be done?
(My original 3 dim matrix is large)
Thanks!
The built-in triu can't deal with this 3D array but you could do it in a simple loop.
for k = 2:size(C, 3)
C(1:k-1,:,k) = 0;
end
You can generate a 2D mask, permute its dimensions, and multiply with singleton expansion using bsxfun:
result = bsxfun(#times, C, permute((1:size(C,1)).'>=(1:size(C,2)), [1 3 2]));
Or, from version R2016b onwards, you can remove bsxfun thanks to automatic singleton expansion:
result = C .* permute((1:size(C,1)).'>=(1:size(C,2)), [1 3 2]);
Assuming you have a "square" 3-D matrix (i.e. NxNxN as you had it in your original post 3x3x3) you could also use reshaping and repmating:
Edit: since repmat is too slow, I swapped it for a bsxfun implementation.
[a,b,c] = size(C)
D = reshape(tril(ones(a)),[a,1,a]);
F = ones(1,size(E,1));
D = bsxfun(#times,D,F);
C(~D)=0;
I also did a quick comparison of the three proposed solutions. #LuisMendo's solution wouldn't work for me, there is a matrix dimension error in the >= comparison (sizes [1 N] and [N 1] are compared).
Between my and #Suever's solution, his is significantly faster:
Comparing the three methods with variable size Cs:
Suever's version (for loop):
Took 0.3529s to compute.
Took 0.0002s to compute size 3x3x3.
Took 0.0008s to compute size 10x10x10.
Took 0.0008s to compute size 50x50x50.
Took 0.0455s to compute size 250x250x250.
Took 0.3055s to compute size 500x500x500.
My version (reshape/repmat):
Took 0.9086s to compute.
Took 0.0522s to compute size 3x3x3.
Took 0.0042s to compute size 10x10x10.
Took 0.0017s to compute size 50x50x50.
Took 0.1060s to compute size 250x250x250.
Took 0.7445s to compute size 500x500x500.
suppose there are arrays A and B, both of which can have any arbitrary numbers and size. for example
A=[1 2 3]
B=[4 8 52 7 10]
i was wondering if there was any way to check if any of the elements in A are contained in B without using a loop? Once again, the numbers and size of the array will be arbitrary so i can't hard code it. Any help and input would be appreciated. Thanks!
You mean like this:
A=[1 2 3]; B=[4 8 52 7 10]
ismember(A,B)
ans =
0 0 0
Add to #NasserM.Abbasi: ismember will work regardless shapes and sizes, it is always element-wise. So if you have A=[1 2; 3 4] and B=[1 3; 4 2; 1 2],
then ismember will return :
ismember(A,B); % -> [1 1; 1 1] due to element-wise membership check
ismember(A,B, 'rows'); % -> [1 0] since it checks row-wise.
use ismember carefully according to the possible shapes in A and B.