I have a multi dimensional array in julia:
julia> ac.value
3x100x3 Array{Float64,3}:
[:, :, 1] =
0.29238 0.0751815 0.00843636 … -0.0143826 0.0403283 0.0225896
0.263146 0.080687 0.000462262 -0.00635778 0.0307563 0.0379104
0.992458 0.986423 0.980587 0.561173 0.55516 0.549105
[:, :, 2] =
0.362155 0.13406 0.0741124 … 0.0231614 0.0156455 0.0121797
0.325581 0.11181 0.0447847 0.0098042 0.0193873 0.0146943
0.914888 0.852297 0.796608 -0.0500265 -0.0551787 -0.0520171
[:, :, 3] =
0.269976 0.108082 0.0441809 … 0.0249861 0.0128778 0.0168318
0.218475 0.0997567 0.0532782 0.0243412 0.00742072 0.00978782
0.96878 0.947455 0.931407 0.0796884 0.0710757 0.0630705
When I look at
julia> ac.value[1,:,1]
1x100 Array{Float64,2}:
0.29238 0.0751815 0.00843636 … -0.0143826 0.0403283 0.0225896
I get a 2 dimensional array, but when I look at
julia> ac.value[:,1,1]
3-element Array{Float64,1}:
0.29238
0.263146
0.992458
I get a one dimensional array. Why is this so and what can I do so that in the former case I get a one-dimensional array?
A quick hack to get 1D arrays is to append [:] to your array references. E.g.,
julia> A = rand(Int,3,3,3)
3x3x3 Array{Int32,3}:
[:, :, 1] =
1059011904 -1092196516 -2083742447
-1232110963 46419394 599245389
747779547 1800837260 -460798437
[:, :, 2] =
-154984919 1641929284 1335793910
-1575337246 1100743707 333491108
231729201 1543773782 338937245
[:, :, 3] =
-1812252712 374672056 -156561770
317145782 -1941995702 747015018
127966143 -102265949 1068453724
julia> A[:,1,1]
3-element Array{Int32,1}:
1059011904
-1232110963
747779547
julia> A[:,1,1][:]
3-element Array{Int32,1}:
1059011904
-1232110963
747779547
julia> A[1,:,1]
1x3 Array{Int32,2}:
1059011904 -1092196516 -2083742447
julia> A[1,:,1][:]
3-element Array{Int32,1}:
1059011904
-1092196516
-2083742447
julia> A[1,1,:]
1x1x3 Array{Int32,3}:
[:, :, 1] =
1059011904
[:, :, 2] =
-154984919
[:, :, 3] =
-1812252712
julia> A[1,1,:][:]
3-element Array{Int32,1}:
1059011904
-154984919
-1812252712
You may also be interested in the ArrayViews package which may be more efficient depending on what you want to implement.
Related
Suppose in Julia you have a function which outputs a vector, like this:
foo(x,y) = rand(3)
And you broadcast it over some arrays
xs = [i for i=1:4, j=1:4]
ys = [j for i=1:4, j=1:4]
A = foo.(xs,ys)
Now A is an array of arrays. How can I turn this into a 3 dimensional array? More generally, is there a solution for higher dimensional situations? E.g. for a similar function foo that outputs an n dimensional array, broadcast over m dimensional arrays xs and ys, how can one convert the broadcasted output to an n+m dimensional array?
I've tried this solution, but vcat and hcat only work on one dimension. I've tried the Destruct.jl package, but again it only works on one dimension. I've tried all the obvious indexing tricks I can think of, but e.g. (foo.(xs,ys))[:,:][1] just returns (foo.(xs,ys))[1], which is not what I want.
You can use SplitApplyCombine.jl like this:
julia> using SplitApplyCombine
julia> A
4×4 Matrix{Vector{Float64}}:
[0.718738, 0.911636, 0.0113524] [0.356858, 0.454262, 0.599563] … [0.455822, 0.475587, 0.567987]
[0.126141, 0.435478, 0.854416] [0.621569, 0.210589, 0.404811] [0.682072, 0.245013, 0.962997]
[0.361148, 0.753464, 0.752801] [0.0643847, 0.193872, 0.820067] [0.676852, 0.942302, 0.093561]
[0.0223175, 0.0972373, 0.215464] [0.737345, 0.737404, 0.996896] [0.183609, 0.335617, 0.720999]
julia> combinedims(A)
3×4×4 Array{Float64, 3}:
[:, :, 1] =
0.718738 0.126141 0.361148 0.0223175
0.911636 0.435478 0.753464 0.0972373
0.0113524 0.854416 0.752801 0.215464
[:, :, 2] =
0.356858 0.621569 0.0643847 0.737345
0.454262 0.210589 0.193872 0.737404
0.599563 0.404811 0.820067 0.996896
[:, :, 3] =
0.135787 0.851451 0.930372 0.498012
0.74397 0.552808 0.960158 0.506592
0.502612 0.0145137 0.915655 0.538791
[:, :, 4] =
0.455822 0.682072 0.676852 0.183609
0.475587 0.245013 0.942302 0.335617
0.567987 0.962997 0.093561 0.720999
Is this what you wanted to get?
I assume that you want to concatenate such that for the result B, we have
B[i,j,:] == A[i,j]
Here's how I'd write this by hand, with a couple of test cases:
julia> function stack_inner(A)
m = ndims(A)
n = ndims(eltype(A))
s_outer = size(A)
s_inner = size(A[begin])
T = eltype(eltype(A))
B = similar(A, T, s_outer..., s_inner...)
.. = fill(:, n)
for ix in CartesianIndices(A)
view(B, ix, (..)...) .= A[ix]
end
return B
end
stack_inner (generic function with 1 method)
julia> stack_inner(foo.(xs, ys))
4×4×3 Array{Float64, 3}:
[:, :, 1] =
0.146902 0.454526 0.191392 0.0322454
0.824091 0.482875 0.700646 0.431301
0.701653 0.0762824 0.194861 0.62421
0.663212 0.0853607 0.313588 0.368867
[:, :, 2] =
0.828077 0.74424 0.255279 0.666977
0.854976 0.302373 0.649691 0.41975
0.759374 0.200208 0.502984 0.886694
0.315596 0.683564 0.956973 0.170769
[:, :, 3] =
0.765305 0.327369 0.824123 0.0537041
0.642428 0.595402 0.235029 0.53
0.603463 0.87867 0.913007 0.548221
0.058201 0.0320288 0.636 0.39045
julia> A = foo.(xs,ys)
4×4 Matrix{Vector{Float64}}:
[0.369792, 0.692929, 0.330885] [0.986054, 0.628871, 0.604634] [0.367734, 0.974091, 0.621425] [0.848115, 0.76681, 0.070687]
[0.00585058, 0.0253985, 0.0470831] [0.10664, 0.373489, 0.111656] [0.719105, 0.126512, 0.660547] [0.999209, 0.0836153, 0.56231]
[0.88527, 0.745378, 0.380452] [0.861579, 0.252228, 0.303043] [0.506468, 0.645717, 0.443472] [0.322553, 0.80937, 0.90342]
[0.783752, 0.553846, 0.830212] [0.868647, 0.0431845, 0.868717] [0.533789, 0.247143, 0.968839] [0.813371, 0.78052, 0.0166259]
julia> stack_inner(A)
4×4×3 Array{Float64, 3}:
[:, :, 1] =
0.369792 0.986054 0.367734 0.848115
0.00585058 0.10664 0.719105 0.999209
0.88527 0.861579 0.506468 0.322553
0.783752 0.868647 0.533789 0.813371
[:, :, 2] =
0.692929 0.628871 0.974091 0.76681
0.0253985 0.373489 0.126512 0.0836153
0.745378 0.252228 0.645717 0.80937
0.553846 0.0431845 0.247143 0.78052
[:, :, 3] =
0.330885 0.604634 0.621425 0.070687
0.0470831 0.111656 0.660547 0.56231
0.380452 0.303043 0.443472 0.90342
0.830212 0.868717 0.968839 0.0166259
julia> xs = rand(2,2,3); ys = rand(2,2,3);
julia> A = foo.(xs, ys)
2×2×3 Array{Vector{Float64}, 3}:
[:, :, 1] =
[0.992239, 0.0603648, 0.687885] [0.896318, 0.825111, 0.31629]
[0.348628, 0.580808, 0.365436] [0.31947, 0.126727, 0.364692]
[:, :, 2] =
[0.714268, 0.0538692, 0.404262] [0.1527, 0.556172, 0.922746]
[0.521115, 0.383689, 0.731707] [0.663383, 0.764024, 0.61838]
[:, :, 3] =
[0.23384, 0.388472, 0.413886] [0.119036, 0.117612, 0.365978]
[0.208551, 0.875924, 0.783887] [0.444344, 0.466899, 0.523953]
julia> stack_inner(A)
2×2×3×3 Array{Float64, 4}:
[:, :, 1, 1] =
0.992239 0.896318
0.348628 0.31947
[:, :, 2, 1] =
0.714268 0.1527
0.521115 0.663383
[:, :, 3, 1] =
0.23384 0.119036
0.208551 0.444344
[:, :, 1, 2] =
0.0603648 0.825111
0.580808 0.126727
[:, :, 2, 2] =
0.0538692 0.556172
0.383689 0.764024
[:, :, 3, 2] =
0.388472 0.117612
0.875924 0.466899
[:, :, 1, 3] =
0.687885 0.31629
0.365436 0.364692
[:, :, 2, 3] =
0.404262 0.922746
0.731707 0.61838
[:, :, 3, 3] =
0.413886 0.365978
0.783887 0.523953
julia> foo(x,y) = rand(3, 3)
foo (generic function with 1 method)
julia> xs = rand(1,2); ys = rand(1,2);
julia> A = foo.(xs, ys)
1×2 Matrix{Matrix{Float64}}:
[0.886281 0.898241 0.0377659; 0.720996 0.401784 0.567878; 0.670619 0.681678 0.457421] [0.0157844 0.23003 0.155695; 0.698028 0.535197 0.458348; 0.500317 0.880123 0.970028]
julia> stack_inner(A)
1×2×3×3 Array{Float64, 4}:
[:, :, 1, 1] =
0.886281 0.0157844
[:, :, 2, 1] =
0.720996 0.698028
[:, :, 3, 1] =
0.670619 0.500317
[:, :, 1, 2] =
0.898241 0.23003
[:, :, 2, 2] =
0.401784 0.535197
[:, :, 3, 2] =
0.681678 0.880123
[:, :, 1, 3] =
0.0377659 0.155695
[:, :, 2, 3] =
0.567878 0.458348
[:, :, 3, 3] =
0.457421 0.970028
(The funny .. is just a hack to do what one could use EllipsisNotation.jl for.)
I'm attempting to use findall to get an index of which elements of one 1d array are greater than those of a second 1d array, and then use those indexes to set corresponding values of a third 1d array to 0.
MWE:
# create 3d array
a, b = [3;2;2], [4;3;2];
c = transpose(cat(a,b, dims = 2));
d, e = [1;2;3], [2;3;4];
f = transpose(cat(d,e, dims = 2));
g = cat(c, f, dims = 3);
g
2×3×2 Array{Int64,3}:
[:, :, 1] =
3 2 2
4 3 2
[:, :, 2] =
1 2 3
2 3 4
findall.(g[end,:,1] >= g[end-1,:,1])
and use indexes to reset elements of g[end,:,2] such that I end up with
g
2×3×2 Array{Int64,3}:
[:, :, 1] =
3 2 2
4 3 2
[:, :, 2] =
1 2 3
0 0 4
Thx. J
The code below gives the answer you request. You just have the . in the wrong spot. You want to compare the > operation element by element, and then apply findall to the entire resulting array (not element by element).
julia> g[end, findall(g[end,:,1] .> g[end-1,:,1]), 2] .= 0
2-element view(::Array{Int64,3}, 2, [1, 2], 2) with eltype Int64:
0
0
julia> g
2×3×2 Array{Int64,3}:
[:, :, 1] =
3 2 2
4 3 2
[:, :, 2] =
1 2 3
0 0 4
However, I wouldn't try to compile all your data into one big array like that. It would be easier to use three separate 1D array variables than three dimensions in one variable. Again using your variables above:
julia> e[b .> a] .= 0
2-element view(::Array{Int64,1}, [1, 2]) with eltype Int64:
0
0
julia> e
3-element Array{Int64,1}:
0
0
4
My old friend the 3d array called Pop, I want to remove columns (d2), across all of d3 when the value in the end of d1 == 1, so I have this code:
Pop[end, :, 1] .!=1
I thought adding #view in front of this would write the changes back to Pop, outwith producing an additional copy in memory. This code works fine
#view(Pop[ :, Pop[end, :, 1] .!=1, :])
but it does not alter the original 3d array called Pop. I could do
Pop = #view(Pop[ :, Pop[end, :, 1] .!=1, :])
but I believe this will create another copy in memory which I'm trying to avoid. What simple syntax have I missed? Thx. J
#view does not modify the size of the original array, it provides a "view" into it, (e.g., omitting some columns in your case). I don't think there is anything wrong with
Pop = view(Pop, :, Pop[end, :, 1] .≠ 1, :)
since now Pop is a view into your old, full Pop, but it behaves like an array, so you can modify its entries, e.g., you could then do things like
julia> using Random # using a fixed seed for reproducibility
julia> Random.seed!(0) ;
julia> Pop = rand(1:5, (2,4,2)) # original Pop
2×4×2 Array{Int64,3}:
[:, :, 1] =
4 3 5 5
1 1 3 5
[:, :, 2] =
2 2 3 1
2 5 1 1
julia> Pop[end,:,1] .≠ 1 # columns to keep
4-element BitArray{1}:
0
0
1
1
julia> Pop = view(Pop, :, Pop[end, :, 1] .≠ 1, :) # make it a view
2×2×2 view(::Array{Int64,3}, :, [3, 4], :) with eltype Int64:
[:, :, 1] =
5 5
3 5
[:, :, 2] =
3 1
1 1
julia> Pop[end,:,1] .= 1 ; # use your new view to manipulate data
julia> Pop # view of the modified Pop
2×2×2 view(::Array{Int64,3}, :, [3, 4], :) with eltype Int64:
[:, :, 1] =
5 5
1 1
[:, :, 2] =
3 1
1 1
julia> Pop.parent # original full Pop (now Pop.parent) has been modified
2×4×2 Array{Int64,3}:
[:, :, 1] =
4 3 5 5
1 1 1 1
[:, :, 2] =
2 2 3 1
2 5 1 1
How do I zip a two-dimensional array with a "vector" row-wise in Julia?
This
X = [1 2; 3 4]
ndims(X)
Y = [-1 -2]
ndims(Y)
first(zip(X,Y))
gives (1, -1) while I want to get ([1 2], -1).
If you're ok with using column-vectors for the input and output, then you can use the eachrow function, which iterates over the rows of a matrix and returns the rows as column-vectors:
julia> X = [1 2; 3 4];
julia> Y = [-1, -2];
julia> collect(zip(eachrow(X), Y))
2-element Array{Tuple{Array{Int64,1},Int64},1}:
([1, 2], -1)
([3, 4], -2)
On the other hand, if you need the first elements of your zipped tuples to be row-vectors (as is shown in your question), then you could convert your matrix into a vector of rows and then use zip:
julia> X = [1 2; 3 4];
julia> Y = [-1 -2];
julia> rows = [X[[i], :] for i in 1:size(X, 1)]
2-element Array{Array{Int64,2},1}:
[1 2]
[3 4]
julia> collect(zip(rows, Y))
2-element Array{Tuple{Array{Int64,2},Int64},1}:
([1 2], -1)
([3 4], -2)
Note that I've used X[[i], :] inside the comprehension instead of X[i, :], so that we get an array of rows rather than an array of column-vectors.
Finally, just to be clear, note that Y = [-1 -2] produces a row-vector. We usually represent vectors as column vectors:
julia> Y = [-1, -2]
2-element Array{Int64,1}:
-1
-2
There are iterator builders in Julia: eachrow and eachcol, which work for arrays and are concise (at least in this case):
X = [1 2; 3 4]
Y = [-1 -2]
z = zip(eachrow(X), eachcol(Y))
Then
for el in z
print(el)
end
gives
([1, 2], [-1])
([3, 4], [-2])
I have an array of arrays A which is an N-element Array{Array{Int64,1},1} of integers. I am trying to find the largest array in A using Julia.
For example:
A = [[1, 2], [3, 4], [5, 6, 7], [1, 2, 5, 8]]
In Python I would simply do: max(A, key=len) but in Julia I don't know how to do it.
What I did is this:
L = []
for a in A
push!(L, length(a))
end
A[findmax(L)[2]]
Thanks!
#Colin has provided a compact, convenient answer. However, if speed matters (op asked for most efficient way) this should be close to optimum
function findlongest(A)
idx = 0
len = 0
#inbounds for i in 1:length(A)
l = length(A[i])
l > len && (idx = i; len=l)
end
return A[idx]
end
Note that this implementation would (presumably) be a really bad idea in Python :)
Quick benchmark:
julia> using BenchmarkTools
julia> A = [[1,2], [1,2,3,4,5,6], [1,2,3]]
3-element Array{Array{Int64,1},1}:
[1, 2]
[1, 2, 3, 4, 5, 6]
[1, 2, 3]
julia> #btime findlongest(A);
26.880 ns (0 allocations: 0 bytes)
julia> #btime A[indmax(length.(A))];
9.813 μs (25 allocations: 1.14 KiB)
That's a ~365 times speedup for this example.
EDIT: Better benchmark (suggested in the comments)
julia> #btime findlongest($A);
9.813 ns (0 allocations: 0 bytes)
julia> #btime $A[indmax(length.($A))];
41.813 ns (1 allocation: 112 bytes)
The $ signs avoid setup allocations and times. Speedup ~4.
Quick explanation
for loops are fast in Julia, so why not use them
avoid allocation (length.(A) allocates a new array of integers)
a && b is shortcut for "if a then b"
#inbounds avoids bound checks for A[i]
UPDATE: For v1+ you'll need to replace indmax in this answer with argmax.
EDIT: Note, it is also worth checking out the other answer by #crstnbr
Consider the following example code:
julia> A = [[1,2], [1,2,3,4,5,6], [1,2,3]]
3-element Array{Array{Int64,1},1}:
[1, 2]
[1, 2, 3, 4, 5, 6]
[1, 2, 3]
julia> length(A)
3
julia> length.(A)
3-element Array{Int64,1}:
2
6
3
julia> indmax(length.(A))
2
julia> A[indmax(length.(A))]
6-element Array{Int64,1}:
1
2
3
4
5
6
The first call to length gets the length of the outer vector in A, which is not what we want. In the second call, I use the broadcasting operator . so that I instead get the length of each of the inner vectors. In the indmax line, I'm finding the index of largest value in length.(A), ie the index of the longest inner vector. If you instead want to return the longest inner vector, you can just index into A using the result of the indmax line.
indmax is no longer defined in Julia (at least 1.3).
Use argmax instead.
>>> A = [[1,2], [1,2,3]]
2-element Array{Array{Int64,1},1}:
[1, 2]
[1, 2, 3]
>>> length.(A)
2-element Array{Int64,1}:
2
3
>>> argmax(length.(A))
2
>>> A[argmax(length.(A))]
3-element Array{Int64,1}:
1
2
3