I have a nonlinear equation that uses an initial y'(a) and outputs a value y(b) such that y(b)=f(y'(a)), where f(x) is some function. The idea is that I'd like to be able to maximize y(b).
Typically, if I had a value for y(b), I could use the shooting or secant method. However I don't have that value. I was thinking I could use a loop to find the max value, but that is very inefficient. Anything better I could use?
*Edit: Also I do not have an explicit expression for f(x).
Thanks,
Mike
Related
The N2 diagram for my full problem is below.
The N2 diagram for the coupled portion of the problem is below.
I have a DirectSolver handling the coupling between LLTForces and ImplicitLiftingLine, and an LNBGS solver handling the coupling between LiftingLineGroup and TestCL.
The gist for the problem is here: https://gist.github.com/eufren/31c0e569ed703b2aea3e2ef5360610f7
I have implemented guess_nonlinear() on ImplicitLiftingLine, which should use various outputs from LLTGeometry to give a good initial guess for the vortex strengths based on a linearised form of the governing equations.
def guess_nonlinear(self, inputs, outputs, resids):
freestream_unit_vector = inputs['freestream_unit_vector']
freestream_velocity = inputs['freestream_velocity']
n = inputs['normal_vectors']
A = inputs['surface_areas']
l = inputs['bound_vortices']
ic_tot = inputs['influence_coefficients_total']
v_inf = freestream_velocity
v_inf_vec = v_inf*freestream_unit_vector
lin_numerator = np.pi * v_inf * A * np.sum(n * v_inf_vec, axis=1)
lin_denominator = (np.linalg.norm(np.cross(v_inf_vec, l), axis=1) - np.pi * v_inf * A * np.sum(np.sum(n * ic_tot, axis=2), axis=1))
lin_vtx_str = lin_numerator / lin_denominator
outputs['vortex_strengths'] = lin_vtx_str
However, when the problem is run for the first time, any inputs not explicitly set with p.set_val() are all 1s. This causes guess_nonlinear() to give a bad output and so the system fails to converge:
As far as I can tell, the execution order for the LLT group is correct, and the geometry components should be being executed before the implicit component. I'm confused as to why this doesn't seem to actually be happening when the code is run, and instead these inputs are taking their default values.
What do I need to change to get this to work properly? Additionally, I've found difficulty in getting LNBGS to converge (hence adding guess_nonlinear()) during optimisation - only DirectSolver gets all the way through the optimisation without issues, but it's very slow for large numbers of LLT nodes). How can I improve the linear and nonlinear solver selection, and improve the reliability of the iterative solver?
Note: Thanks for providing a testable example. It made figuring out the answer to your question a lot simpler. Your problem was a bit subtle and I would not have been able to give a good answer without runnable code
Your first question: "Why are all the inputs 1"
"Short" Answer
You have put the nonlinear solver to high in the model hierarchy, which then included a key precurser component that computed your input values. By moving the solver down to a lower level of the model, I was able to ensure that the precurser component (LTTGeometry) ran and had valid outputs before you got to the guess_nonlinear of implicit component.
Here is what you had (Notice the implicit solver included LTTGeometry even though the data cycle does not require that component:
I moved both the nonlinear solver and the linear solver down into the LTTCycle group, which then allows the LTTGeometry component to execute before getting to the nonlinear solver and guess_nonlinear step:
My fix is only partially correct, since there is a secondary cycle from the TestCL component that also needs a solver and does not have one. However, that cycle still does not involve the LTTGeometry group. So the fully correct fix is to restructure you model top run geometry first, and then put the LTTCycle and TestCL groups together so you can run a solver over just them. That was a bit more hacking than I wanted to do on your test problem, but you can see the general idea from the adjusted N2 above.
Long Answer
The guess_nonlinear sequence in OpenMDAO does NOT run the compute method of explicit components or of groups. It follows the execution hierarchy, and calls any guess_nonlinear that it finds. So that means that any explicit components you have in your model will NOT get executed, their outputs will not get updated with computed values, and those computed values will not get passed to the inputs of downstream components.
Things get a little tricky when you have deep model hierarchies. The guess_nonlinear method is called as the first step in the nonlinear solver process. If you have a NonLinearRunOnce solver at the top level, it will follow the compute chain down the line calling compute or solve_nonlinear on each child and doing a data transfer after each one. If one of those children happens to be a group with a nonlinear solver, then that solver will call guess_nonlinear on its children (grandchildren of the top group with the NonLinearRunOnce solver) as the first step. So any outputs that were computed by the siblings of this group will be valid, but none of the outputs from the grandchild level will have been computed yet.
You may be wondering why not just have the guess_nonlinear method call the compute for any explicit components? There is a difficult to balance trade off here. If you assume that all explicit components are very cheap to run, then it might make sense to run the compute methods --- or it might not. A lot depends on the cyclic data structure. If some early component in the group needs guesses from the later one, then running its compute isn't going to help you much at all. Perhaps more importantly though, not all explicit components are cheap to run. You might have a very expensive computation, and calling compute as part of the guess process would be way too costly.
The compromise here, if you need some kind of top level guess process, is that you can implement guess_nonlinear at the group level. It's less common to do, but it gives you total control over what happens. You can call whatever you need to call in whatever sequence.
So the absolute key thing to remember is that the only data you have available to you when a guess_nonlinear is called is any data that was computed before your containing solver was executed. That means any thing that was computed before you got to the model scope of the containing solver (not the scope of the component with the guess_method itself).
Your second question: "How can I speed this up when the number of nodes gets large?"
This one not possible to give a generic answer to at all. I noticed that you have already specified sparse partial derivatives. That is a great start, but if its still not fast enough for you then it means you're reaching the limits of what you can do with a DirectSolver. You note that this solver is the only one that gets you through the optimization without issues, which I will take to mean that ScipyKryloventer link description here and PetscKrylov are not converging the linear system well for you --- at least not by themselves. Thats not surprising, as krylov solvers almost always require some kind of preconditioner... and this is why I can't offer a generic answer. Setting up efficient linear solvers for larger-scale compute is a tricky subject. If you look into the literature, you'll find some good suggestions. You can also study open source implementations like VSPAero for some tips.
effectively, you've reached the limit of what simple linear solvers can offer you. From this point forward, OpenMDAO can help a bit by making it easier to implement some preconditioning, but you'll have to suffer the math side yourself.
I want to know if there is a way in Tensorflow's seq2seq framework where I can know if a reply to an input can be given with x% of confidence.
An example below:
I have hi as reply to hello. It works fine. I also have bunch of other trained sentences. However, let's say I enter some junk like this - sdjshj sdjk oiqwe qw. Seq2seq still tries to give a response. I understand it designed that way, but I want to know if there is a way which says the framework cannot answer this with confidence. Or no such words were trained.
This would be of great help.
Use logistic function (or sigmoid) on the output logits:
Because logit function is basically the inverse of sigmoid function:
Logit Function:
Sigmoid Function:
You can see that it is similar. In tensorflow. there is the sigmoid function, but I find the program is faster when you just code the sigmoid function:
If you use sigmoid function. You will get a value from 0 to 1 which is the confidence you are looking for. More information can be found here:
https://en.wikipedia.org/wiki/Sigmoid_function
https://en.wikipedia.org/wiki/Logit
I think average perplexityreturned by seq2seq_model.model.stop is the confidence, the smaller, the better. But it could be hard for one to tell a proper threshhold.
I am trying to create a vector dynamically in dependence of n (for example 1 or 4). If my n is bigger I need to have more values in my vector.
for i=1:(N-n)
yvecT(i)=y(n+i); % Achtung, Zeilenvektor
for k=n:-1:1
F(i-1+n,:)=[-y(i) -y(i-k) u(i) u(i-k)];
end
end
%n=1 F(i,:)=[-y(i) u(i)];
%n=2 F(i,:)=[-y(i) -y(i-1) u(i) u(i-1)];
%n=4 F(i,:)=[-y(i) -y(i-1) -y(i-2) -y(i-3) u(i) u(i-1) u(i-2) u(i-3)];
it is a function used to identify a System....
You should have posted the for-loop (with the if-statements) from the link in the question and stated that you wanted it to work for an arbitary n. That would have made everyone understand your problem. I think the easiest way to do what you do is to use subreferencing. So in case n==2 we do not have
F(i-1,:)=[-y(i) -y(i-1) u(i) u(i-1)];
but rather,
F(i-(n-1),:)=[-y(i:-1:(n-1)) u(i:-1:(n-1))];
This looks messier, but it works for any arbitary n. Some other comments about the code. The variable i is also a function returning the imaginary unit. By naming a variable i you overload this function. The recommended way is to use 1i as an imaginary unit, so it is not critical, but in case you do not necessarily need i as a variable you should consider another name. Also it is easier for us to understand in case you write in english. So in general, prefer comments in english when posting here.
I am currently trying to write a code to solve a non linear system of equations. I am using the functions of the gsl library, more specifically the multiroot_fdf_solver.
My problem is that it currently doesn't want to converge. More specifically, I have the following behavior:
-if my initial conditions are close to the result, the gsl_multiroot_fdf_solver_iterate does not update the parameters at all. I tried to display the results on the different steps, and I have for all the parameters dx = NaN (I think this quite srange), the status of gsl_multiroot_fdf_solver_iterate is "success" and the status of gsl_multiroot_test_residual is "the iteration has not converged yet"
-the parameters are only updated if my initial conditions are really far from the expected result. Obvisously in this case it does not converge to the right values.
I have already checked multiple times the expression of my function and my Jacobian, and they seem good.
I have to precise that my Jacobian (and my system as well) are quite complicated expression with many trigonometric function.
Would you have any idea of what it could be? Is it possible that if the expression of the Jacobian is too complicated, it has troubles to compute it?
Thank you in advance for your answers, I am really stucked at this point.
While trying to make a program for hidden markov models, I did the simplest assumption for the initial HMM of the Baum-Welch algorithm : put everything as a uniform distribution. That is,
A[i][j] = 1/statenumber;
B[i][j] = 1/observationnumber;
P[i] = 1/statenumber;
up to a logarithm to avoid underflowing. It has the benefit of not requiring to check for normalization.
But so far, I've run into the algorithm not actually doing much. The emission matrix changes at the first iteration, but not after that, and the transition matrix and initialization vector do not evolve at all. It seems to be that the gamma matrix does not change at all.
At first I thought it was my algorithm not working out too well, but after trying it on some other HMM libraries, I seem get the same type of results.
Is it impossible to converge to the correct HMM using such an initialization, and what is the ideal method to initialize those arrays?
The Baum Welch algorithm won't work with a uniform initial distribution -- the updates will be degenerate. Try to randomize it instead.