I want to create a 2D list that can have elements of variable lengths inside, for example, if I have a 10x10 list in MATLAB, I can
define it with:
z = cell(10,10)
and start assigning some elements by doing this:
z{2}{3} = ones(3,1)
z{1}{1} = zeros(100,1)
z{1}{2} = []
z{1}{3} = randn(20,1)
...
What is the optimal way to define such empty 2D list in torch? Moreover, is there a way to exploit the tensor structure to do this?
In python, I can do something along this to define an empty 10x10 2D list:
z = [[None for j in range(10)] for i in range(10)]
My best guess for torch is doing something like
z = torch.Tensor(10,10)
for i=1,10 do
for j=1,10 do
z[{{i},{j}}] = torch.Tensor()
end
end
but, this does not work, and defining a tensor inside a tensor seems like a bad idea ...
This is a follow up to the question asked here (however in the link it is asked in python): Create 2D lists in python with variable length indexed vectors
From the documentation I've read, tensors only support primitive numeric data types. You won't be able to use tensor for your intended usage. Leverage tables.
local function makeMatrix(initialVal, ...)
local isfunc = type(initialVal) == "function"
local dimtable = {...}
local function helper(depth)
if depth == 0 then
return isfunc and initialVal() or initialVal
else
local plane = {}
for i = 1, dimtable[depth] do
plane[i] = helper(depth-1)
end
return plane
end
end
return helper(#dimtable)
end
p = makeMatrix(0, 2, 3, 5) -- makes 3D matrix of size 2x3x5 with all elements initialized to 0
makeMatrix(torch.Tensor, m ,n)
Answer from Torch's Google Group forums. Agreeing that tables is the solution:
z = {}
for i=1,10 do
z[i] = {}
for j=1,10 do
z[i][j] = torch.Tensor()
end
end
Related
With the Julia Language, I defined a function to sample points uniformly inside the sphere of radius 3.14 using rejection sampling as follows:
function spherical_sample(N::Int64)
# generate N points uniformly distributed inside sphere
# using rejection sampling:
points = pi*(2*rand(5*N,3).-1.0)
ind = sum(points.^2,dims=2) .<= pi^2
## ideally I wouldn't have to do this:
ind_ = dropdims(ind,dims=2)
return points[ind_,:][1:N,:]
end
I found a hack for subsetting arrays:
ind = sum(points.^2,dims=2) .<= pi^2
## ideally I wouldn't have to do this:
ind_ = dropdims(ind,dims=2)
But, in principle array indexing should be a one-liner. How could I do this better in Julia?
The problem is that you are creating a 2-dimensional index vector. You can avoid it by using eachrow:
ind = sum.(eachrow(points.^2)) .<= pi^2
So that your full answer would be:
function spherical_sample(N::Int64)
points = pi*(2*rand(5*N,3).-1.0)
ind = sum.(eachrow(points.^2)) .<= pi^2
return points[ind,:][1:N,:]
end
Here is a one-liner:
points[(sum(points.^2,dims=2) .<= pi^2)[:],:][1:N, :]
Note that [:] is dropping a dimension so the BitArray can be used for indexing.
This does not answer your question directly (as you already got two suggestions), but I rather thought to hint how you could implement the whole procedure differently if you want it to be efficient.
The first point is to avoid generating 5*N rows of data - the problem is that it is very likely that it will be not enough to generate N valid samples. The point is that the probability of a valid sample in your model is ~50%, so it is possible that there will not be enough points to choose from and [1:N, :] selection will throw an error.
Below is the code I would use that avoids this problem:
function spherical_sample(N::Integer) # no need to require Int64 only here
points = 2 .* pi .* rand(N, 3) .- 1.0 # note that all operations are vectorized to avoid excessive allocations
while N > 0 # we will run the code until we have N valid rows
v = #view points[N, :] # use view to avoid allocating
if sum(x -> x^2, v) <= pi^2 # sum accepts a transformation function as a first argument
N -= 1 # row is valid - move to the previous one
else
rand!(v) # row is invalid - resample it in place
#. v = 2 * pi * v - 1.0 # again - do the computation in place via broadcasting
end
end
return points
end
This one is pretty fast, and uses StaticArrays. You can probably also implement something similar with ordinary tuples:
using StaticArrays
function sphsample(N)
T = SVector{3, Float64}
v = Vector{T}(undef, N)
n = 1
while n <= N
p = rand(T) .- 0.5
#inbounds v[n] = p .* 2π
n += (sum(abs2, p) <= 0.25)
end
return v
end
On my laptop it is ~9x faster than the solution with views.
Let's say I have an array of vectors:
""" simple line equation """
function getline(a::Array{Float64,1},b::Array{Float64,1})
line = Vector[]
for i=0:0.1:1
vector = (1-i)a+(i*b)
push!(line, vector)
end
return line
end
This function returns an array of vectors containing x-y positions
Vector[11]
> Float64[2]
> Float64[2]
> Float64[2]
> Float64[2]
.
.
.
Now I want to seprate all x and y coordinates of these vectors to plot them with plotyjs.
I have already tested some approaches with no success!
What is a correct way in Julia to achive this?
You can broadcast getindex:
xs = getindex.(vv, 1)
ys = getindex.(vv, 2)
Edit 3:
Alternatively, use list comprehensions:
xs = [v[1] for v in vv]
ys = [v[2] for v in vv]
Edit:
For performance reasons, you should use StaticArrays to represent 2D points. E.g.:
getline(a,b) = [(1-i)a+(i*b) for i=0:0.1:1]
p1 = SVector(1.,2.)
p2 = SVector(3.,4.)
vv = getline(p1,p2)
Broadcasting getindex and list comprehensions will still work, but you can also reinterpret the vector as a 2×11 matrix:
to_matrix{T<:SVector}(a::Vector{T}) = reinterpret(eltype(T), a, (size(T,1), length(a)))
m = to_matrix(vv)
Note that this does not copy the data. You can simply use m directly or define, e.g.,
xs = #view m[1,:]
ys = #view m[2,:]
Edit 2:
Btw., not restricting the type of the arguments of the getline function has many advantages and is preferred in general. The version above will work for any type that implements multiplication with a scalar and addition, e.g., a possible implementation of immutable Point ... end (making it fully generic will require a bit more work, though).
for the following code:
from array import *
x=[]
x.append(0.232)
print (x)
for i in range(25):
x[i+1]=(1/(i+1))-5*x[i]
I have this error:
x[i+1]=(1/(i+1))-5*x[i]
IndexError: list assignment index out of range
This may be happening because I have defined x to be an empty array. But how do I define the array and perform the same operation otherwise?
list is not designed for efficient mathematical operations and therefore its better to use numpy arrays for doing mathematical operations. However, if you want to use list, you may define a list initialized with n zero's using
x=[0]*n
x[0] = 0.232
x[1] = ....
....
Remember, that a multidimensional list created using above approach will refer to same element in the array! For example:
l = [0,0,0]*5
will be creating five same list's inside another list not separate list's. So its a bad idea to create multidimensional array like this!
A better way would be to create arrays using numpy using following code:
from numpy import empty, zeros
x = empty(n) # or # x = zeros(n)
x[0] = 0.232
x[1] = ....
....
and
l = empty((3,5)) # or # l = zeros((3,5))
for a array with 3 rows and 5 columns.
my program in R creates an n-dimensional array.
PVALUES = array(0, dim=dimensions)
where dimensions = c(x,y,z, ... )
The dimensions will depend on a particular input. So, I want to create a general-purpose code that will:
Store a particular element in the array
Read a particular element from the array
From reading this site I learned how to do #2 - read an element from the array
ll=list(x,y,z, ...)
element_xyz = do.call(`[`, c(list(PVALUES), ll))
Please help me solving #1, that is storing an element to the n-dimensional array.
Let me rephrase my question
Suppose I have a 4-dimensional array. I can store a value and read a value from this array:
PVALUES[1,1,1,1] = 43 #set a value
data = PVALUES[1,1,1,1] #use a value
How can I perform the same operations using a function of a vector of indexes:
indexes = c(1,1,1,1)
set(PVALUES, indexes) = 43
data = get(PVALUES, indexes) ?
Thank you
Thanks for helpful response.
I will use the following solution:
PVALUES = array(0, dim=dimensions) #Create an n-dimensional array
dimensions = c(x,y,z,...,n)
Set a value to PVALUES[x,y,z,...,n]:
y=c(x,y,z,...,n)
PVALUES[t(y)]=26
Reading a value from PVALUES[x,y,z,...,n]:
y=c(x,y,z,...,n)
data=PVALUES[t(y)]
The indexing of arrays can be done with matrices having the same number of columns as there are dimensions:
# Assignment with "[<-"
newvals <- matrix( c( x,y,z,vals), ncol=4)
PVALUES[ newvals[ ,-4] ] <- vals
# Reading values with "["
PVALUES[ newvals[ ,-4] ]
I need the best way to store a three dimensional table for pixels. What I need to do is have multiple x,y tables (basically three dimensional) it is to raster multiple two dimensional pixel maps with transparency. You see I can create two dimensions easily like so:
pixels = {{},{}}
pixels[1][5] = "green" --just an example
print(pixels[1][5])
However, I cannot do this like I can in Java...
pixels = {{}, {}, {}}
pixels[1][4][3] = "red" -- [x][y][z]
print(pixels[1][4][3])
This is the functionality I want, but I have disgustingly got around this by doing this...
pixels = {}
pixels["x23,y02,z05"] = "green"
print(pixels["x23,y02,z05"]")
I just use string.sub, and string.concat to read and set the tables... I really would like the functionality of example 2 to work, however I know it might need to be implemented differently.
There are basically two ways to go about this
Auto-tables
Auto-tables generate sub-tables transparently using metatables and essentially after creating it you should be able to forget about them.
function newAutotable(dim)
local MT = {};
for i=1, dim do
MT[i] = {__index = function(t, k)
if i < dim then
t[k] = setmetatable({}, MT[i+1])
return t[k];
end
end}
end
return setmetatable({}, MT[1]);
end
-- Usage
local at = newAutotable(3);
print(at[0]) -- returns table
print(at[0][1]) -- returns table
print(at[0][1][2]) -- returns nil
at[0][1][2] = 2;
print(at[0][1][2]) -- returns value
print(at[0][1][3][3]) -- error, because only 3 dimensions set
What is not so nice about them is that they generate lots of tables -- obviously. That's some memory overhead and each level of depth increases the execution time.
What's nice about them is that they can be completely dynamic in size. You could even make them infinitely deep. Though in your use-case this is very likely not necessary and probably even a bad idea.
This structure is very suitable for non-integer indexes though, you could for example make the depth even depend on a "template structure" and so implement a transparent dynamic configuration table, but I'm getting side-tracked...
Flattened arrays
The other variant are flattened arrays. user3125367 already wrote about them, but I want to expand on this as this can be done a lot more convenient and explain a few things.
Often flattening your multi-dimensional arrays is a good idea in CG anyway, since then you can do many matrix operations very easily. Calculating a modified index is also relatively cheap in terms of processing time required. But it should be noted, although kind of obvious, that this approach only works with numeric keys and a predefined size of your matrix.
function newMdArray(X, Y, Z)
local MT = { __call = function(t, x, y, z, v)
if x>X or y>Y or z>Z or x<1 or y<1 or z<1 then return; end
local k = x + X*(y-1) + X*Y*(z-1);
if v ~= nil then t[k] = v; end
return t[k];
end };
return setmetatable({}, MT);
end
-- Usage
local mdt = newMdArray(100, 100, 100);
local v = mdt(1, 2, 3);
mdt(1, 2, 3, v*.1);
This code is taken from another answer from me: dynamic tables or arrays
It can probably be optimised a little (for example calculate X*Y in the closure) but I wanted to paste the original code here. Anyway, with this you can both easily work on the flattened structure by just using normal array indexing:
for i=1, #mdt
mdt[i] = (mdt[i] or 0)*.5
end
As well as access 3d indexes directly:
mdt(12, 13, 14, 0)
You can also easily modify the function to return a default value for missing keys by adding an __index field to the metatable or so that the table saves the matrix dimensions etc.
In addition to classic 'array in array in array' scheme, you can use benefits of Lua table internals. How? Lua table is just a mapping from key to value, and when you use it as an array, you may skip some keys and this will cost virtually nothing.
t = { }
t[1] = "Hello"
t[500000] = "World" -- does NOT allocate additional 499999 elements
So, if your data is sparse (over 50% of your 3d-points having no value), you may benefit from this:
local n_x, n_y, n_z = 1920, 1080, 1000
local n_xy = n_x * n_y
function setValue(t, x, y, z, value)
assert(x > 0 and x < n_x)
assert(y > 0 and y < n_y)
assert(z > 0 and z < n_z)
t[((z-1) * n_xy) + ((y-1) * n_z) + x] = value
end
function getValue(t, x, y, z)
assert(x > 0 and x < n_x)
assert(y > 0 and y < n_y)
assert(z > 0 and z < n_z)
return t[((z-1) * n_xy) + ((y-1) * n_z) + x]
end
t = { }
setValue(t, 1, 1, 1, "red")
setValue(t, 1, 1, 2, "green")
In your first code:
pixels = {{},{}}
is equivalent to:
pixels = {}
pixels[1] = {}
pixels[2] = {}
Here, pixels[1] is already a table, that's why you can assign a value to pixels[1][5].
But in you second code:
pixels = {{}, {}, {}}
Here, pixels is still a two-dimensional array (with 3 elements). It's equivalent to :
pixels = {}
pixels[1] = {}
pixels[2] = {}
pixels[3] = {}
pixels[1] is a table, but pixels[1][4] is not. What you need to do is to give pixels[1][4] a table constructor like this:
pixels = {{}, {}, {}}
pixels[1][4] = {} --initialize it to an empty table
pixels[1][4][3] = "red"
print(pixels[1][4][3])