Is it possible to interleave two arrays in julia?
For example if a=[1:10] and b=[11:20] I want to be able to return
20-element Array{Int64,1}:
1
11
2
12
3
13
4
14
.
.
.
Somewhat similar to what ruby can do Merge and interleave two arrays in Ruby
There is a straightforward way to do this without needing to use the reshape() function. In particular, we can just bind the vectors into a matrix and then use [:] on the transpose of that matrix. For example:
julia> a = 1:10
julia> b = 11:20
julia> [a b]'[:]
20-element Array{Int64,1}:
1
11
2
12
3
13
.
.
.
20
Taking the transpose of the matrix [a b] gives us a 2-by-10 matrix, and then [:] returns all of its elements in the form of a vector. The reason [:] works so nicely for us is because Julia uses column-major ordering.
Figured it out!
reshape([a b]',20,1)
and for something more general:
reshape([a b].',size(a,1)+size(b,1),1)
we can use a hack to get vectors instead of 1D arrays:
reshape([a b].',size(a,1)+size(b,1),1)[:]
You could just use
reshape([a b].', length(a)+length(b))
to get a vector.
julia> #benchmark collect(Iterators.flatten(zip(a,b))) setup = begin a=rand(100); b=rand(100) end
BenchmarkTools.Trial: 10000 samples with 714 evaluations.
Range (min … max): 190.895 ns … 1.548 μs ┊ GC (min … max): 0.00% … 65.85%
Time (median): 238.843 ns ┊ GC (median): 0.00%
Time (mean ± σ): 265.428 ns ± 148.757 ns ┊ GC (mean ± σ): 8.40% ± 11.75%
▅▅██▅▃▂▂▁ ▁▁ ▂
██████████▇█▇▇▅▄▄▃▁▁▁▃▄█▅▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▃▁▁▁▁▃▃▁▁▁▁▁▃▁▃▄▆▇███ █
191 ns Histogram: log(frequency) by time 1.11 μs <
Memory estimate: 1.77 KiB, allocs estimate: 1.
it seems that
collect(Iterators.flatten(zip(a,b)))
is much faster
For completeness, expanding #bdeonovic's solution to 2 dimensional arrays.
julia> a
2×2 Array{Int64,2}:
1 2
3 4
julia> b
2×2 Array{Int64,2}:
6 7
8 9
Interweaving rows:
julia> reshape([a[:] b[:]]', 4, 2)
4×2 Array{Int64,2}:
1 2
6 7
3 4
8 9
Interweaving columns:
julia> reshape( [a' b']', 2, 4 )
2×4 Array{Int64,2}:
1 6 2 7
3 8 4 9
Interweaving arrays (stacking/vcatting):
julia> reshape([a' b']', 4, 2)
4×2 Array{Int64,2}:
1 2
3 4
6 7
8 9
Related
Related:
How to convert from string to array?
This is a follow-up question. How would I make a list of all the digits in this number (currently as a string)?
"123" -> [1,2,3]
There are no delimiters here so how should I go about doing this?
Note as of now I am using the latest version of Julia, v1.8.3 so parse doesn't seem to work in the other question's answers. Error when I use parse():
ERROR: LoadError: MethodError: no method matching parse(::SubString{String})
Closest candidates are:
parse(::Type{T}, ::AbstractString) where T<:Complex at parse.jl:381
parse(::Type{Sockets.IPAddr}, ::AbstractString) at ~/usr/share/julia/stdlib/v1.8/Sockets/src/IPAddr.jl:246
parse(::Type{T}, ::AbstractChar; base) where T<:Integer at parse.jl:40
...
Stacktrace:
[1] iterate
# ./generator.jl:47 [inlined]
[2] _collect
# ./array.jl:807 [inlined]
[3] collect_similar
# ./array.jl:716 [inlined]
[4] map
# ./abstractarray.jl:2933 [inlined]
[5] top-level scope
# ~/proc/self/fd/0:1
in expression starting at /proc/self/fd/0:1
exit status 1
Easy peasy like this:
function str2vec(s::String)
return map(x->parse(Int,x), split(s,""))
end
julia> str2vec("124")
3-element Vector{Int64}:
1
2
4
Or by broadcasting:
julia> parse.(Int, split("124",""))
3-element Vector{Int64}:
1
2
4
By piping functions:
julia> "124" |> x->split(x, "") |> x->parse.(Int, x)
3-element Vector{Int64}:
1
2
4
Utilizing the eachsplit function, which is a lazy function and returns a generator object (introduced in Julia 1.8):
julia> eachsplit("124", "") |> x->parse.(Int, x)
3-element Vector{Int64}:
1
2
4
According to Dan's advice, you try another ways:
Using the Int8 on the collected chars:
julia> Int8.(collect("124")).-48
3-element Vector{Int64}:
1
2
4
Using the Iterators.map:
julia> collect(Iterators.map(x->Int8(x)-48,"124"))
3-element Vector{Int64}:
1
2
4
Also, one can consider the DNF's proposal:
julia> [Int(x)-48 for x in "124"]
3-element Vector{Int64}:
1
2
4
Benchmarking
julia> using BenchmarkTools
julia> #btime str2vec("124");
#btime parse.(Int, split("124",""));
#btime "124" |> x->split(x, "") |> x->parse.(Int, x);
#btime eachsplit("124", "") |> x->parse.(Int, x);
#btime Int8.(collect("124")).-48;
#btime collect(Iterators.map(x->Int8(x)-48,"123"));
#btime [Int(x)-48 for x in "123"]
681.250 ns (11 allocations: 864 bytes)
675.460 ns (11 allocations: 864 bytes)
679.747 ns (11 allocations: 864 bytes)
1.280 μs (14 allocations: 816 bytes)
92.412 ns (2 allocations: 160 bytes)
61.711 ns (1 allocation: 80 bytes)
45.152 ns (1 allocation: 80 bytes)
You can also use the inbuilt digits function.
By default, it returns the digits last-to-first:
julia> digits(parse(Int, "1234"))
4-element Vector{Int64}:
4
3
2
1
You can reverse! the result if you want them in the same order as in the string:
julia> digits(parse(Int, "1234")) |> reverse!
4-element Vector{Int64}:
1
2
3
4
This runs much faster than parseing each digit individually. The Int8(...) .- 48 method is still faster, but it fails silently if the input string happens to be invalid, which could be dangerous further down the line. Since we're using parse here, this method reports the error correctly in such cases.
julia> Int8.(collect("invalid")).-48
7-element Vector{Int64}:
57
62
70
49
60
57
52
julia> digits(parse(Int, "invalid")) |> reverse!
ERROR: ArgumentError: invalid base 10 digit 'i' in "invalid"
Both other answers are very good, but they have forgotten about comprehensions. Using a comprehension gives both the fastest safe solution, and the absolute fastest solution, tied with the Iterators.map.
Fastest unsafe (based on the answer by #Shayan with input from #DanGetz):
julia> #btime [Int(c)-48 for c in "123"]
34.372 ns (1 allocation: 80 bytes)
3-element Vector{Int64}:
1
2
3
The above will silently return the wrong answer for invalid inputs, as noted by #SundarR.
Here's an even nicer and more intuitive version of the above, which is the same under the hood:
[c - '0' for c in "123"]
It works because Int('0') equals 48, and subtraction of Chars yields an Int.
Fastest safe solution (based on #SundarR's answer):
julia> #btime [parse(Int, c) for c in "123"]
47.822 ns (1 allocation: 80 bytes)
3-element Vector{Int64}:
1
2
3
julia> [parse(Int, c) for c in "invalid"]
ERROR: ArgumentError: invalid base 10 digit 'i'
I would probably recommend the latter in most cases.
One more thing you may or may not be aware of: You can create a generator instead of a vector, in case you don't actually need the vector itself, but want to iterate over the converted numbers for some other purpose. The syntax is almost identical to an array comprehension, just use () instead:
g = (parse(Int, c) for c in "123")
for val in g
println(val, " squared equals ", val^2)
end
1 squared equals 1
2 squared equals 4
3 squared equals 9
This will not allocate an intermediate temporary vector, and creating the generator is essentially free:
julia> #btime (parse(Int, c) for c in "123")
1.900 ns (0 allocations: 0 bytes)
The computational cost is paid during iteration instead. This is similar to using Iterators.map without collect, but arguably has nicer syntax.
I want to turn a array of arrays into a matrix. To illustrate; let the array of arrays be:
[ [1,2,3], [4,5,6], [7,8,9]]
I would like to turn this into the 3x3 matrix:
[1 2 3
4 5 6
7 8 9]
How would you do this in Julia?
There are several ways of doing this. For instance, something along the lines of vcat(transpose.(a)...) will work as a one-liner
julia> a = [[1,2,3], [4,5,6], [7,8,9]]
3-element Vector{Vector{Int64}}:
[1, 2, 3]
[4, 5, 6]
[7, 8, 9]
julia> vcat(transpose.(a)...)
3×3 Matrix{Int64}:
1 2 3
4 5 6
7 8 9
though note that
Since your inner arrays are column-vectors as written, you need to transpose them all before you can vertically concatenate (aka vcat) them (either that or horizontally concatenate and then transpose the whole result after, i.e., transpose(hcat(a...))), and
The splatting operator ... which makes this one-liner work will not be very efficient when applied to Arrays in general, and especially not when applied to larger arrays-of-arrays.
Performance-wise for larger arrays-of-arrays, it will likely actually be hard to beat preallocating a result of the right size and then simply filling with a loop, e.g.
result = similar(first(a), length(a), length(first(a)))
for i=1:length(a)
result[i,:] = a[i] # Aside: `=` is actually slightly faster than `.=` here, though either will have the same practical result in this case
end
Some quick benchmarks for reference:
julia> using BenchmarkTools
julia> #benchmark vcat(transpose.($a)...)
BechmarkTools.Trial: 10000 samples with 405 evaluations.
Range (min … max): 241.289 ns … 3.994 μs ┊ GC (min … max): 0.00% … 92.59%
Time (median): 262.836 ns ┊ GC (median): 0.00%
Time (mean ± σ): 289.105 ns ± 125.940 ns ┊ GC (mean ± σ): 2.06% ± 4.61%
▁▆▇█▇▆▅▅▅▄▄▄▄▃▂▂▂▃▃▂▂▁▁▁▂▄▃▁▁ ▁ ▁ ▂
████████████████████████████████▇▆▅▆▆▄▆▆▆▄▄▃▅▅▃▄▆▄▁▃▃▃▅▄▁▃▅██ █
241 ns Histogram: log(frequency) by time 534 ns <
Memory estimate: 320 bytes, allocs estimate: 5.
julia> #benchmark for i=1:length($a)
$result[i,:] = $a[i]
end
BechmarkTools.Trial: 10000 samples with 993 evaluations.
Range (min … max): 33.966 ns … 124.918 ns ┊ GC (min … max): 0.00% … 0.00%
Time (median): 36.710 ns ┊ GC (median): 0.00%
Time (mean ± σ): 39.795 ns ± 7.566 ns ┊ GC (mean ± σ): 0.00% ± 0.00%
▄▄██▄▅▃ ▅▃ ▄▁▂ ▂▁▂▅▂▁ ▄▂▁ ▂
██████████████▇██████▆█▇▆███▆▇███▇▆▆▅▆▅▅▄▄▅▄▆▆▆▄▁▃▄▁▃▄▅▅▃▁▄█ █
34 ns Histogram: log(frequency) by time 77.7 ns <
Memory estimate: 0 bytes, allocs estimate: 0.
In general, filling column-by-column (if possible) will be faster than filling row-by-row as we have done here, since Julia is column-major.
Expanding on #cbk's answer, another (slightly more efficient) one-liner is
julia> transpose(reduce(hcat, a))
3×3 transpose(::Matrix{Int64}) with eltype Int64:
1 2 3
4 5 6
7 8 9
[1 2 3; 4 5 6; 7 8 9]
# or
reshape(1:9, 3, 3)' # remember that ' makes the transpose of a Matrix
Basically I have two 1d numpy arrays, let's call them x and y, both of the same length. I want to essentially get the result x1y1 + x2y2 + ... + xn*yn. Obviously I could do this with a for loop but is there a built-in method or something where I can do this in one line?
What you are trying to compute is known as an 'inner product' and, in the case of two vectors, is called a 'dot product'. Numpy has built-in functions for computing both which are optimized for speed over the simple (x*y).sum() solution.
import numpy as np
a = np.array([1, 2, 3])
b = np.array([3, 2, 1])
print(np.inner(a, b))
# 10
print(np.dot(a, b))
# 10
Some timing results in the table below with vectors a and b being 1000 randomly selected elements using np.random.randn:
np.dot(a, b) # 920 ns ± 9.9 ns
np.inner(a, b) # 1.1 µs ± 83.5 ns
(a*b).sum() # 4.2 µs ± 62.9 ns
np.sum(a*b) # 5.7 µs ± 170 ns
You can use sum(x*y) or (x*y).sum(), they're equivalent.
In Octave, I can do
octave:1> A = [1 2; 3 4]
A =
1 2
3 4
octave:2> A(A>1) -= 1
A =
1 1
2 3
but in Julia, the equivalent syntax does not work.
julia> A = [1 2; 3 4]
2x2 Array{Int64,2}:
1 2
3 4
julia> A[A>1] -= 1
ERROR: `isless` has no method matching isless(::Int64, ::Array{Int64,2})
in > at operators.jl:33
How do you conditionally assign values to certain array or matrix elements in Julia?
Your problem isn't with the assignment, per se, it's that A > 1 itself doesn't work. You can use the elementwise A .> 1 instead:
julia> A = [1 2; 3 4];
julia> A .> 1
2×2 BitArray{2}:
false true
true true
julia> A[A .> 1] .-= 1000;
julia> A
2×2 Array{Int64,2}:
1 -998
-997 -996
Update:
Note that in modern Julia (>= 0.7), we need to use . to say that we want to broadcast the action (here, subtracting by the scalar 1000) to match the size of the filtered target on the left. (At the time this question was originally asked, we needed the dot in A .> 1 but not in .-=.)
In Julia v1.0 you can use the replace! function instead of logical indexing, with considerable speedups:
julia> B = rand(0:20, 8, 2);
julia> #btime (A[A .> 10] .= 10) setup=(A=copy($B))
595.784 ns (11 allocations: 4.61 KiB)
julia> #btime replace!(x -> x>10 ? 10 : x, A) setup=(A=copy($B))
13.530 ns ns (0 allocations: 0 bytes)
For larger matrices, the difference hovers around 10x speedup.
The reason for the speedup is that the logical indexing solution relies on creating an intermediate array, while replace! avoids this.
A slightly terser way of writing it is
replace!(x -> min(x, 10), A)
There doesn't seem to be any speedup using min, though.
And here's another solution that is almost as fast:
A .= min.(A, 10)
and that also avoids allocations.
To make it work in Julia 1.0 one need to change = to .=. In other words:
julia> a = [1 2 3 4]
julia> a[a .> 1] .= 1
julia> a
1×4 Array{Int64,2}:
1 1 1 1
Otherwise you will get something like
ERROR: MethodError: no method matching setindex_shape_check(::Int64, ::Int64)
I have a function, which returns a two dimensional Array:
2-element Array{Float64,1}:
0.809919
2.00754
I now want to efficiently sample over it and store all the results in an array with 2 rows and n columns. The problem is that I get a Vector of vectors. How could I flatten it or construct it?
A toy example is the following:
julia> [rand(2) for i=1:3]
3-element Array{Array{Float64,1},1}:
[0.906644, 0.614673]
[0.426492, 0.67645]
[0.473704, 0.726284]
julia> [rand(2)' for i=1:3]
3-element Array{RowVector{Float64,Array{Float64,1}},1}:
[0.403384 0.431918]
[0.410625 0.546614]
[0.224933 0.118778]
And I would like to have the result in a form like this:
julia> [rand(2) rand(2) rand(2)]
2×3 Array{Float64,2}:
0.360833 0.205969 0.209643
0.507417 0.317295 0.588516
Actually my dream would be:
julia> [rand(2) rand(2) rand(2)]'
3×2 Array{Float64,2}:
0.0320955 0.821869
0.358808 0.26685
0.230355 0.31273
Any ideas? I know that I could construct it via a for loop, but was looking for a more efficient way.
Thanks!
RecursiveArrayTools.jl has a VectorOfArray type which dispatches in the way you'd want:
julia> using RecursiveArrayTools
julia> A = [rand(2) for i=1:3]
3-element Array{Array{Float64,1},1}:
[0.957228, 0.104218]
[0.293985, 0.83882]
[0.788157, 0.454772]
julia> VectorOfArray(A)'
3×2 Array{Float64,2}:
0.957228 0.104218
0.293985 0.83882
0.788157 0.454772
As for timing:
julia> #benchmark VectorOfArray(A)'
BenchmarkTools.Trial:
memory estimate: 144 bytes
allocs estimate: 2
--------------
minimum time: 100.658 ns (0.00% GC)
median time: 111.740 ns (0.00% GC)
mean time: 127.159 ns (3.29% GC)
maximum time: 1.360 μs (82.71% GC)
--------------
samples: 10000
evals/sample: 951
VectorOfArray itself is almost no overhead, and the ' uses the Cartesian indexing to be fast.
Something along these lines
using BenchmarkTools
function createSample!(vec::AbstractVector)
vec .= randn(length(vec))
return vec
end
function createSamples!(A::Matrix)
for row in indices(A, 1)
createSample!(view(A, row, :))
end
return A
end
A = zeros(10, 2)
#benchmark createSamples!(A)
might help. The timing on my laptop gives:
Main> #benchmark createSamples!(A)
BenchmarkTools.Trial:
memory estimate: 1.41 KiB
allocs estimate: 20
--------------
minimum time: 539.104 ns (0.00% GC)
median time: 581.194 ns (0.00% GC)
mean time: 694.601 ns (13.34% GC)
maximum time: 10.324 μs (90.10% GC)
--------------
samples: 10000
evals/sample: 193