In my Julia optimisation model, I have a variable A which is a 5x5x5 array. Then A[:,:,1] is a matrix. In my model, I wish to calculate the sum of the column vectors of A[:,:,1]. Here is my code:
I = 1:5; J = 1:5; K = 1:5;
m = Model(HiGHS.Optimizer)
#variable(m, A[i in I, j in J, t in K] >= 0)
sum(A[:,:,1],dims = 2)
However, this gives me an error:
No method is implemented for reducing index range of type UnitRange{Int64}. Please implement reduced_index for this index type or report this as an issue.
How can I fix this error? The code works if I write
#variable(m, A[i in 1:5, j in 1:5, t in 1:5] >= 0)
but in my model, the indices I, J and K are given as an input to a function, so I cannot specify them like that.
A[:,:,1] yields a 2-dimensional DenseAxisArray. You should do collect on it to be able to aggregate via sum(..., dims=...). Hence you need to do:
julia> sum(collect(A[:,:,1]),dims = 2)
5×1 Matrix{AffExpr}:
A[1,1,1] + A[1,2,1] + A[1,3,1] + A[1,4,1] + A[1,5,1]
A[2,1,1] + A[2,2,1] + A[2,3,1] + A[2,4,1] + A[2,5,1]
A[3,1,1] + A[3,2,1] + A[3,3,1] + A[3,4,1] + A[3,5,1]
A[4,1,1] + A[4,2,1] + A[4,3,1] + A[4,4,1] + A[4,5,1]
A[5,1,1] + A[5,2,1] + A[5,3,1] + A[5,4,1] + A[5,5,1]
Please also note that your variable definition can be shortened to #variable(m, A[I, J, K] >= 0)
Finally, see my other recommendation on your model layout at your second question:
Summing columns of an array in model constraint in Julia
Related
Consider the following optimisation model:
I = 1:5; J = 1:5; K = 1:5;
m = Model(HiGHS.Optimizer)
#variable(m, A[i in I, j in J, k in K] >= 0)
#variable(m, y[i in I, k in K] >= 0)
#constraint(m, sum(collect(A[:,:,1]),dims = 2) .== y[:,1])
In the model, I have a 5x5x5 array variable A, and a 5x5 array variable y. I want to have an equation constraint, where the vector obtained by adding the column vectors of the matrix A[:,:,1] together is (entry-wise) equal to the column vector p[:,1].
However, the above code gives an error:
MethodError: no method matching _broadcasted_type(::Base.Broadcast.ArrayStyle{JuMP.Containers.DenseAxisArray}, ::Base.HasShape{2}, ::Type{AffExpr})
How can I fix that?
Apply collect to the other side of the equation:
julia> #constraint(m, vec(sum(collect(A[:,:,1]),dims = 2)) .== collect(y[:,1]))
5-element Vector{ConstraintRef{Model, MathOptInterface.ConstraintIndex{MathOptInterface.ScalarAffineFunction{Float64}, MathOptInterface.EqualTo{Float64}}, ScalarShape}}:
A[1,1,1] + A[1,2,1] + A[1,3,1] + A[1,4,1] + A[1,5,1] - y[1,1] == 0.0
A[2,1,1] + A[2,2,1] + A[2,3,1] + A[2,4,1] + A[2,5,1] - y[2,1] == 0.0
....
EDIT
note that your JuMP experience will be much more enjoyable if you do:
I = 5; J = 5; K = 5;
#variable(m, A[1:I, 1:J, 1:K] >= 0)
#variable(m, y[1:I, 1:K] >= 0)
Now you do not have dense arrays and can do what you initially wanted:
julia> #constraint(m, sum(A[:,:,1],dims = 2) .== y[:,1])
5×1 Matrix{ConstraintRef{Model, MathOptInterface.ConstraintIndex{MathOptInterface.ScalarAffineFunction{Float64}, MathOptInterface.EqualTo{Float64}}, ScalarShape}}:
A[1,1,1] + A[1,2,1] + A[1,3,1] + A[1,4,1] + A[1,5,1] - y[1,1] == 0.0
A[2,1,1] + A[2,2,1] + A[2,3,1] + A[2,4,1] + A[2,5,1] - y[2,1] == 0.0
...
Im writing currently rewriting a Matlab script in C. When i get to the last few lines of the Matlab script a for loop is executed and it iterates through an array. Since i am trying to rewrite the program in C the slicing notation in the Matlab script is confusing me. I have attached the line of code that is troubling me below.
How would i write this line of code in a nested for loop indexing with i and j only, since you cant slice in c obviously. just for reference u = 1, Ubc is 2D array of size (NX+2, NY+2). Where NX = NY = 40.
Below is the line of code in Matlab i need to translate to for loop indexing.
Nx = 40;
Ny = 40;
u = 1;
Ubc = rand(Nx + 2, Ny + 2);
% First the i interfaces
F = 0.5* u *( Ubc(2:Nx+2,2:Ny+1) + Ubc(1:Nx+1,2:Ny+1))
- 0.5*abs(u)*( Ubc(2:Nx+2,2:Ny+1) - Ubc(1:Nx+1,2:Ny+1));
You can calculate the same in a loop as
Nx = 40;
Ny = 40;
u = 1;
Ubc = rand(Nx + 2, Ny + 2);
F = zeros(Nx + 1, Ny);
for z1 = 1 : Nx + 1
for z2 = 1 : Ny
F(z1, z2) = 0.5* u *( Ubc(z1 + 1, z2 + 1) + Ubc(z1, z2 + 1))
- 0.5*abs(u)*( Ubc(z1 + 1, z2 + 1) - Ubc(z1, z2 + 1));
end
end
You shouldn't use i and j as loop index in Matlab. Both are the imaginary unit.
We are given array of 'n' integers and we are given queries in the form (l,r) where l and r are indices in range 'n'. For each query the answer is Suppose the array is a={a1,a2,a3,a4,a5,a6,a7...}
and the query is (2,7) then for this query it should give a2*a7+a3*a6+a4*a5
This means that first element is multiplied with the last in the query range, second is multiplied by second last element and so on.
Length of each query is divisible by 2
Is there any way to do this using segment tree>
Here's an O(k n log n + q (n/k))-time solution (so if q = Θ(n) we set k = √(n/log n) to get O(n √(n log n))).
The key ingredient is a fast convolution algorithm, perhaps based on FFT, though per djb and probably others, in the n = 1e5 range, you might get better results from an asymptotically slower algorithm. If we convolve the input array with itself, we get (e.g., for a 9-element array):
c2 = a1*a1
c3 = a1*a2 + a2*a1
c4 = a1*a3 + a2*a2 + a3*a1
c5 = a1*a4 + a2*a3 + a3*a2 + a4*a1
c6 = a1*a5 + a2*a4 + a3*a3 + a4*a2 + a5*a1
c7 = a1*a6 + a2*a5 + a3*a4 + a4*a3 + a5*a2 + a6*a1
c8 = a1*a7 + a2*a6 + a3*a5 + a4*a4 + a5*a3 + a6*a2 + a7*a1
c9 = a1*a8 + a2*a7 + a3*a6 + a4*a5 + a5*a4 + a6*a3 + a7*a2 + a8*a1
c10 = a1*a9 + a2*a8 + a3*a7 + a4*a6 + a5*a5 + a6*a4 + a7*a3 + a8*a2 + a9*a1
c11 = a2*a9 + a3*a8 + a4*a7 + a5*a6 + a6*a5 + a7*a4 + a8*a3 + a9*a2
c12 = a3*a9 + a4*a8 + a5*a7 + a6*a6 + a7*a5 + a8*a4 + a8*a3
c13 = a4*a9 + a5*a8 + a6*a7 + a7*a6 + a8*a5 + a9*a4
c14 = a5*a9 + a6*a8 + a7*a7 + a8*a6 + a9*a5
c15 = a6*a9 + a7*a8 + a8*a7 + a9*a6
c16 = a7*a9 + a8*a8 + a9*a7
c17 = a8*a9 + a9*a8
c18 = a9*a9
Already the odd coefficients are closely related to some of the possible answers to queries (e.g., c9/2 is the answer to (1,8)).
Our approach is to compute the self-convolution of k-1 prefixes of the array and k-1 suffixes (actually we only need the odd coefficients, not that this is an asymptotic speedup), i.e., a[1..n/k], a[1..2n/k], ..., a[1..(k-1)n/k]; a[n/k+1..n], a[2n/k+1..n], ..., a[(k-1)n/k+1..n]. To answer a query (l,r), we select a good subarray, grab the self-convolution coefficient at index l+r, divide it by two, and fix it up by adding O(n/k) terms.
Rather than write this precisely in mathematical notation, let me give an example. Suppose n = 9 and k = 3 and we want to answer the query (2,7). We grab the coefficient
c9 = a3*a6 + a4*a5 + a5*a4 + a6*a3
for the subarray a[1..6] and return
c9/2 + a2*a7.
What's the best subarray? If l+r <= n, then we should round r down to r' a multiple of n/k and use a[1..r']. Otherwise we should round l up to l' a multiple of n/k and use a[l'+1..n].
I have the antenna array factor expression here:
I have coded the array factor expression as given below:
lambda = 1;
M = 100;N = 200; %an M x N array
dx = 0.3*lambda; %inter-element spacing in x direction
m = 1:M;
xm = (m - 0.5*(M+1))*dx; %element positions in x direction
dy = 0.4*lambda;
n = 1:N;
yn = (n - 0.5*(N+1))*dy;
thetaCount = 360; % no of theta values
thetaRes = 2*pi/thetaCount; % theta resolution
thetas = 0:thetaRes:2*pi-thetaRes; % theta values
phiCount = 180;
phiRes = pi/phiCount;
phis = -pi/2:phiRes:pi/2-phiRes;
cmpWeights = rand(N,M); %complex Weights
AF = zeros(phiCount,thetaCount); %Array factor
tic
for i = 1:phiCount
for j = 1:thetaCount
for p = 1:M
for q = 1:N
AF(i,j) = AF(i,j) + cmpWeights(q,p)*exp((2*pi*1j/lambda)*(xm(p)*sin(thetas(j))*cos(phis(i)) + yn(q)*sin(thetas(j))*sin(phis(i))));
end
end
end
end
How can I vectorize the code for calculating the Array Factor (AF).
I want the line:
AF(i,j) = AF(i,j) + cmpWeights(q,p)*exp((2*pi*1j/lambda)*(xm(p)*sin(thetas(j))*cos(phis(i)) + yn(q)*sin(thetas(j))*sin(phis(i))));
to be written in vectorized form (by modifying the for loop).
Approach #1: Full-throttle
The innermost nested loop generates this every iteration - cmpWeights(q,p)*exp((2*pi*1j/lambda)*(xm(p)*sin(thetas(j))*cos(phis(i)) + yn(q)*sin(thetas(j))*sin(phis(i)))), which are to summed up iteratively to give us the final output in AF.
Let's call the exp(.... part as B. Now, B basically has two parts, one is the scalar (2*pi*1j/lambda) and the other part
(xm(p)*sin(thetas(j))*cos(phis(i)) + yn(q)*sin(thetas(j))*sin(phis(i))) that is formed from the variables that are dependent on
the four iterators used in the original loopy versions - i,j,p,q. Let's call this other part as C for easy reference later on.
Let's put all that into perspective:
Loopy version had AF(i,j) = AF(i,j) + cmpWeights(q,p)*exp((2*pi*1j/lambda)*(xm(p)*sin(thetas(j))*cos(phis(i)) + yn(q)*sin(thetas(j))*sin(phis(i)))), which is now equivalent to AF(i,j) = AF(i,j) + cmpWeights(q,p)*B, where B = exp((2*pi*1j/lambda)*(xm(p)*sin(thetas(j))*cos(phis(i)) + yn(q)*sin(thetas(j))*sin(phis(i)))).
B could be simplified to B = exp((2*pi*1j/lambda)* C), where C = (xm(p)*sin(thetas(j))*cos(phis(i)) + yn(q)*sin(thetas(j))*sin(phis(i))).
C would depend on the iterators - i,j,p,q.
So, after porting onto a vectorized way, it would end up as this -
%// 1) Define vectors corresponding to iterators used in the loopy version
I = 1:phiCount;
J = 1:thetaCount;
P = 1:M;
Q = 1:N;
%// 2) Create vectorized version of C using all four vector iterators
mult1 = bsxfun(#times,sin(thetas(J)),cos(phis(I)).'); %//'
mult2 = bsxfun(#times,sin(thetas(J)),sin(phis(I)).'); %//'
mult1_xm = bsxfun(#times,mult1(:),permute(xm,[1 3 2]));
mult2_yn = bsxfun(#times,mult2(:),yn);
C_vect = bsxfun(#plus,mult1_xm,mult2_yn);
%// 3) Create vectorized version of B using vectorized C
B_vect = reshape(exp((2*pi*1j/lambda)*C_vect),phiCount*thetaCount,[]);
%// 4) Final output as matrix multiplication between vectorized versions of B and C
AF_vect = reshape(B_vect*cmpWeights(:),phiCount,thetaCount);
Approach #2: Less-memory intensive
This second approach would reduce the memory traffic and it uses the distributive property of exponential - exp(A+B) = exp(A)*exp(B).
Now, the original loopy version was this -
AF(i,j) = AF(i,j) + cmpWeights(q,p)*exp((2*pi*1j/lambda)*...
(xm(p)*sin(thetas(j))*cos(phis(i)) + yn(q)*sin(thetas(j))*sin(phis(i))))
So, after using the distributive property, we would endup with something like this -
K = (2*pi*1j/lambda)
part1 = K*xm(p)*sin(thetas(j))*cos(phis(i));
part2 = K*yn(q)*sin(thetas(j))*sin(phis(i));
AF(i,j) = AF(i,j) + cmpWeights(q,p)*exp(part1)*exp(part2);
Thus, the relevant vectorized approach would become something like this -
%// 1) Define vectors corresponding to iterators used in the loopy version
I = 1:phiCount;
J = 1:thetaCount;
P = 1:M;
Q = 1:N;
%// 2) Define the constant used at the start of EXP() call
K = (2*pi*1j/lambda);
%// 3) Perform the sine-cosine operations part1 & part2 in vectorized manners
mult1 = K*bsxfun(#times,sin(thetas(J)),cos(phis(I)).'); %//'
mult2 = K*bsxfun(#times,sin(thetas(J)),sin(phis(I)).'); %//'
%// Perform exp(part1) & exp(part2) in vectorized manners
part1_vect = exp(bsxfun(#times,mult1(:),xm));
part2_vect = exp(bsxfun(#times,mult2(:),yn));
%// Perform multiplications with cmpWeights for final output
AF = reshape(sum((part1_vect*cmpWeights.').*part2_vect,2),phiCount,[])
Quick Benchmarking
Here are the runtimes with the input data listed in the question for the original loopy approach and proposed approach #2 -
---------------------------- With Original Approach
Elapsed time is 358.081507 seconds.
---------------------------- With Proposed Approach #2
Elapsed time is 0.405038 seconds.
The runtimes suggests a crazy performance improvement with Approach #2!
The basic trick is to figure out what things are constant, and what things depend on the subscript term - and therefore are matrix terms.
Within the sum:
C(n,m) is a matrix
2π/λ is a constant
sin(θ)cos(φ) is a constant
x(m) and y(n) are vectors
So the two things I would do are:
Expand the xm and ym into matrices using meshgrid()
Take all the constant term stuff outside the loop.
Like this:
...
piFactor = 2 * pi * 1j / lambda;
[xgrid, ygrid] = meshgrid(xm, ym); % xgrid and ygrid will be size (N, M)
for i = 1:phiCount
for j = 1:thetaCount
xFactor = sin(thetas(j)) * cos(phis(i));
yFactor = sin(thetas(j)) * sin(phis(i));
expFactor = exp(piFactor * (xgrid * xFactor + ygrid * yFactor)); % expFactor is size (N, M)
elements = cmpWeights .* expFactor; % elements of sum, size (N, M)
AF(i, j) = AF(i, j) + sum(elements(:)); % sum and then integrate.
end
end
You could probably figure out how to vectorise the outer loop too, but hopefully that gives you a starting point.
The below code is supposed to calculate some values and place them in incremental places in the numpy.zeros() array. The calculations all perform correctly but the array stays as just zeros. I could be missing something obvious so apologies if I am.
n = 256
lam = l
a = numpy.zeros([(len(z[0]) * len(z[:,0]) + n + 1), (n + len(z[0]))])
b = numpy.zeros([numpy.size(a, 0), 1])
#data fitting equations
k = 0
for i in range(len(z[0])):
for j in range(len(z[:,0])-1):
wij = smoother(z[j][i] + lam)
a[k][(z[j][i]+lam)] = float(wij)
print a[k][(z[j][i]+lam)]
a[k][n+j] = float(-wij)
b[k][0] = float(-wij * B[j])
k = k + 1
Thanks,
Tom
Answer supplied by Jaime works fine. Use
a[1, 2]
rather than
a[1][2]