Suppose I have two arrays representing a probabilistic graph:
2
/ \
1 -> 4 -> 5 -> 6 -> 7
\ /
3
Where the probability of going to state 2 is 0.81 and the probability of going to state 3 is (1-0.81) = 0.19. My arrays represent the estimated values of the states as well as the rewards. (Note: Each index of the array represents its respective state)
V = [0, 3, 8, 2, 1, 2, 0]
R = [0, 0, 0, 4, 1, 1, 1]
The context doesn't matter so much, it's just to give an idea of where I'm coming from. I need to write a k-step look ahead function where I sum the discounted value of rewards and add it to the estimated value of the kth-state.
I have been able to do this so far by creating separate functions for each step look ahead. My goal of asking this question is to figure out how to refactor this code so that I don't repeat myself and use idiomatic Julia.
Here is an example of what I am talking about:
function E₁(R::Array{Float64,1}, V::Array{Float64, 1}, P::Float64)
V[1] + 0.81*(R[1] + V[2]) + 0.19*(R[2] + V[3])
end
function E₂(R::Array{Float64,1}, V::Array{Float64, 1}, P::Float64)
V[1] + 0.81*(R[1] + R[3]) + 0.19*(R[2] + R[4]) + V[4]
end
function E₃(R::Array{Float64,1}, V::Array{Float64, 1}, P::Float64)
V[1] + 0.81*(R[1] + R[3]) + 0.19*(R[2] + R[4]) + R[5] + V[5]
end
.
.
.
So on and so forth. It seems that if I was to ignore E₁() this would be exceptionally easy to refactor. But because I have to discount the value estimate at two different states, I'm having trouble thinking of a way to generalize this for k-steps.
I think obviously I could write a single function that took an integer as a value and then use a bunch of if-statements but that doesn't seem in the spirit of Julia. Any ideas on how I could refactor this? A closure of some sort? A different data type to store R and V?
It seems like you essentially have a discrete Markov chain. So the standard way would be to store the graph as its transition matrix:
T = zeros(7,7)
T[1,2] = 0.81
T[1,3] = 0.19
T[2,4] = 1
T[3,4] = 1
T[5,4] = 1
T[5,6] = 1
T[6,7] = 1
Then you can calculate the probabilities of ending up at each state, given an intial distribution, by multiplying T' from the left (because usually, the transition matrix is defined transposedly):
julia> T' * [1,0,0,0,0,0,0] # starting from (1)
7-element Array{Float64,1}:
0.0
0.81
0.19
0.0
0.0
0.0
0.0
Likewise, the probability of ending up at each state after k steps can be calculated by using powers of T':
julia> T' * T' * [1,0,0,0,0,0,0]
7-element Array{Float64,1}:
0.0
0.0
0.0
1.0
0.0
0.0
0.0
Now that you have all probabilities after k steps, you can easily calculate expectations as well. Maybe it pays of to define T as a sparse matrix.
Related
I'm trying to simulate a 3D random walk in Julia as a way to learn the ropes of Julia programming. I define all my variables and then initialize an (n_steps X 3) array of zeros that I want to use to store my coordinates when I do the walk. Here, "n_steps" is the number of steps in the walk, and the three columns correspond to the x, y, and z coordinates. When I try to update the array with my new coordinates, I get an error:
ERROR: LoadError: BoundsError: attempt to access 100×3 Array{Float64,2} at index [0, 1]
I don't understand why I'm getting this error. As far as I know, I'm looping through all the rows of the array and updating the x, y, and z coordinates. I never mentioned the index 0, as I specified that the loop start at row number 1 in my code. What is going on? Here is my code so far (I haven't plotted yet, since I can't progress further without resolving this problem):
using Plots
using Random
len_step = 1
θ_min, θ_max = 0, pi
ϕ_min, ϕ_max = 0, 2 * pi
n_steps = 100
init = zeros(Float64, n_steps, 3)
for jj = 1:1:length(init)
θ_rand = rand(Float64)* (θ_max - θ_min)
ϕ_rand = rand(Float64)* (ϕ_max - ϕ_min)
x_rand = len_step * sin(θ_rand) * cos(ϕ_rand)
y_rand = len_step * sin(θ_rand) * sin(ϕ_rand)
z_rand = len_step * cos(θ_rand)
init[jj, 1] += init[jj-1, 1] + x_rand
init[jj, 2] += init[jj-1, 2] + y_rand
init[jj, 3] += init[jj-1, 3] + z_rand
end
print(init)
If it's relevant, I'm running Julia Version 1.4.2 on 64-Bit on Windows 10. I'd greatly appreciate any help. Thanks.
The function length returns the length of an array as if it was one dimensional. What you want is size
julia> init = zeros(3,5)
3×5 Array{Float64,2}:
0.0 0.0 0.0 0.0 0.0
0.0 0.0 0.0 0.0 0.0
0.0 0.0 0.0 0.0 0.0
julia> length(init)
15
julia> size(init)
(3, 5)
julia> size(init, 2)
5
julia> size(init, 1)
3
Note also that in julia, array indices start at 1, and since you access at index jj-1, you can not start the loop at 1.
I am trying out an encoding - decoding method that had been asked in this post
https://stackoverflow.com/questions/40820958/matlab-help-in-implementing-a-mathematical-equation-for-generating-multi-level
and a related one Generate random number with given probability matlab
There are 2 parts to this question - encoding and decoding. Encoding of a symbolic sequence is done using inverse interval mapping using the map f_inv. The method of inverse interval mapping yields a real valued number. Based on the real valued number, we iterate the map f(). The solution in the post in the first link does not work - because once the final interval is found, the iteration of the map f() using the proposed solution does not yield the same exact symbolic array. So, I tried by directly implementing the equations for the forward iteration f() given in the paper for the decoding process, but the decoding does not generate the same symbolic sequence.
Here is a breif explanation of the problem.
Let there be an array b = [1,3,2,6,1] containing N = 5 integer valued elements with probability of occurence of each unique integer as 0.4, 0.2, 0.2, 0.2 respectively. The array b can take any integers from the unique symbol set 1,2,3,4,5,6,7,8. Let n = 8 elements in the symbol set. In essence, the probability for the above data b is
p= [ 0.4 (for symbol 1), 0.2 (for symbol 2) , 0.2 (symbol 3), 0 (for symbol 4 not occuring), 0 (for symbol 5), 0.2(for symbol 6), 0 (for symbol 7), 0 (for symbol 8)]
An interval [0,1] is split into 8 regions. Let, the interval for the data b assumed to be known as
Interval_b = [0, 0.4, 0.6, 0.8, 1];
In general, for n = 8 unique symbols, there are n = 8 intervals such as I_1, I_2, I_3, I_4, I_5, I_6, I_6,I_7,I_8 and each of these intervals is assigned a symbol such as [ 1 2 3 4 5 6 7 8]
Let, x = 0.2848 that has been obtained from the reverse interval mapping for the symbol array b from the solution for the encoding procedure in the link. There is a mapping rule which maps x to the symbol depending on the interval in which x lies and we should obtain the same symbol elements as in b. The rule is
Looks like the argument Interval passed to function ObtainSymbols should contain entries for all elements, including the ones with probability 0. This can be done by adding the statement
Interval = cumsum([0, p_arr]);
immediately before the calls to function ObtainSymbols.
The following is the output with this modificaiton:
...
p_arr = [p_1,p_2,p_3,p_4,p_5,p_6,p_7,p_8];
% unchanged script above this
% recompute Interval for all symbols
Interval = cumsum([0, p_arr]);
% [0 0.4 0.6 0.8 0.8 0.8 1.0 1.0 1.0]
% unchanged script below
[y1,symbol1] = ObtainSymbols(x(1),p_arr,Interval);
[y2,symbol2] = ObtainSymbols(y1,p_arr,Interval);
[y3,symbol3] = ObtainSymbols(y2,p_arr,Interval);
[y4,symbol4] = ObtainSymbols(y3,p_arr,Interval);
[y5,symbol5] = ObtainSymbols(y4,p_arr,Interval);
Symbols = [symbol1,symbol2,symbol3,symbol4,symbol5]
y = [y1,y2,y3,y4,y5]
% Symbols = [1 3 2 6 1]
% y = [0.7136 0.5680 0.8400 0.2000 0.5000]
I have the following variables and code in MATLAB:
A = [ 2 2 2 2 2 ]
B = [ 1 2 3 4 5 ]
B / A % = 1.5
The answer, 1.5 is really the average of the values in B divided by 2, but why does MATLAB do this with these arrays and the / operator?
EDIT:
Looking at the documentation here near the bottom of the page for matrix right-division:
x = B/A where xA = B
...but multiplying x * A above results in [ 3 3 3 3 3 ]. This does not seem to make sense either. Am I missing something here as well?
The right division operator / (or the mrdivide function) will solve a systems of linear equations xA = B for x. From the above documentation link:
If A is a rectangular m-by-n matrix with m ~= n, and B is a matrix with n columns, then x = B/A returns a least-squares solution of the system of equations x*A = B.
It's computing a value for x that best approximates a solution in a least-squares sense, since in this case you have an underdetermined system of equations. There is no exact solution, so x*A doesn't exactly reproduce B.
c = a/b perform right-matrix division. It is not the same as the division of each element.
I am trying to generate "automatically" a vector 0.01, 0.03, 0.1, 0.3, 1, 3, 10, 30 (in multiplicative space).
I know linspace and logspace functions, but I couldn't find any similar function for multiplicative space.
Is there any? Otherwise, how to generate a vector like the one I need?
An easy way with bsxfun, also considering multiplication to smaller spaces:
x = [0.01,0.03,0.05] % initial vector, works for various lengths
n = 12; % times it should get multiplied in rising direction
m = 3; % times it should get multiplied in falling direction
Z = bsxfun( #times, x(:), 10.^(-m:n) )
Z = Z(:)
% if preferred, bulky one-liner:
% Z = reshape( bsxfun( #times, x(:), 10.^(-m:n) ) , 1 , [])
I assumed a multiplication with the multiplication vector, e.g.:
10.^(0:n) = 1 10 100 1000 10000 100000 ....
But custom vectors Y are also possible:
Z = bsxfun( #times, x(:), Y(:)' ) Z = Z(:)
A function that might help you achieving this in a very easy and compact way is the Kronecker tensor product kron.
You can use it to rewrite thewaywewalk's answer as:
v = [0.01;0.03;0.05]; % initial vector
emin = -3; % minimal exponent
emax = 12; % maximal exponent
Z = kron(10.^(emin:emax)',v(:))
which should give you the exact same result.
not very efficient but this will generate what you want. inputvec is your initial vector [0.01 0.03] in this case, multiplier is 10. length of the required string n is 8. n should be a multiple of nn (length of the input vector)
function newvec=multispace(n,inputvec,multiplier)
nn=length(inputvec);
newvec=zeros(1,n);
newvec(1:nn)=inputvec;
for i=1:n/nn-1
newvec(i*nn+1:(i+1)*nn)=(newvec((i-1)*nn+1:(i)*nn)).*multiplier;
end
end
I've been looking for a while onto websearch, however, possibly or probably I am missing the right terminology.
I have arbitrary sized arrays of scalars ...
array = [n_0, n_1, n_2, ..., n_m]
I also have a function f->x->y, with 0<=x<=1, and y an interpolated value from array. Examples:
array = [1,2,9]
f(0) = 1
f(0.5) = 2
f(1) = 9
f(0.75) = 5.5
My problem is that I want to compute the average value for some interval r = [a..b], where a E [0..1] and b E [0..1], i.e. I want to generalize my interpolation function f->x->y to compute the average along r.
My mind boggles me slightly w.r.t. finding the right weighting. Imagine I want to compute f([0.2,0.8]):
array --> 1 | 2 | 9
[0..1] --> 0.00 0.25 0.50 0.75 1.00
[0.2,0.8] --> ^___________________^
The latter being the range of values I want to compute the average of.
Would it be mathematically correct to compute the average like this?: *
1 * (1-0.8) <- 0.2 'translated' to [0..0.25]
+ 2 * 1
avg = + 9 * 0.2 <- 0.8 'translated' to [0.75..1]
----------
1.4 <-- the sum of weights
This looks correct.
In your example, your interval's length is 0.6. In that interval, your number 2 is taking up (0.75-0.25)/0.6 = 0.5/0.6 = 10/12 of space. Your number 1 takes up (0.25-0.2)/0.6 = 0.05 = 1/12 of space, likewise your number 9.
This sums up to 10/12 + 1/12 + 1/12 = 1.
For better intuition, think about it like this: The problem is to determine how much space each array-element covers along an interval. The rest is just filling the machinery described in http://en.wikipedia.org/wiki/Weighted_average#Mathematical_definition .