I want to train a simple circuit in TFQ using a Sequential model as follows:
model = tf.keras.Sequential()
model.add(tf.keras.layers.Input(shape=(), dtype=tf.dtypes.string))
model.add(
tfq.layers.PQC(
model_circuit=circuit,
operators=readout_op))
But instead of performing a readout op, I'd like the model to output the state vector so I can do some post-processing on it before I feed it into my loss function.
In principle, tfq.layers.State looks like it's appropriate for this task, but it is not clear to me from the examples how I would use the State layer in a model context, vs just using it to generate the state vector as shown in the docs:
state_layer = tfq.layers.State()
alphas = tf.reshape(tf.range(0, 1.1, delta=0.5), (3, 1)) # FIXME: #805
state_layer(parametrized_bell_circuit,
symbol_names=[alpha], symbol_values=alphas)
So my questions:
can I force the PQC layer to output the state vector instead of performing a readout operation?
can I use the State layer as a parameterized layer in a Sequential model (or train it in any other way?)
or is there any alternative way that my model outputs a state vector?
can I force the PQC layer to output the state vector instead of performing a readout operation?
The PQC layer will create and manage tf.Variables for you. From there it will send your circuits through a tfq.layers.Expectation layer. Unfortunately there is no way to produce a full state vector from this layer.
can I use the State layer as a parameterized layer in a Sequential model (or train it in any other way?)
Yes, you can incorporate the state vector of input circuits into your model with the tfq.layers.State layer (https://www.tensorflow.org/quantum/api_docs/python/tfq/layers/State). Note that the produced state vector will NOT be differentiable. When creating TFQ we wanted to encourage users doing any complex modelling to try and make use of functionality that would have a 1:1 translation between a real chip and simulation (i.e. It is very easy to deploy tfq.layers.Expectation logic onto a true chip since we aren't breaking any rules, but with tfq.layers.State we are cheating and pulling out the full state vector).
Related
The N2 diagram for my full problem is below.
The N2 diagram for the coupled portion of the problem is below.
I have a DirectSolver handling the coupling between LLTForces and ImplicitLiftingLine, and an LNBGS solver handling the coupling between LiftingLineGroup and TestCL.
The gist for the problem is here: https://gist.github.com/eufren/31c0e569ed703b2aea3e2ef5360610f7
I have implemented guess_nonlinear() on ImplicitLiftingLine, which should use various outputs from LLTGeometry to give a good initial guess for the vortex strengths based on a linearised form of the governing equations.
def guess_nonlinear(self, inputs, outputs, resids):
freestream_unit_vector = inputs['freestream_unit_vector']
freestream_velocity = inputs['freestream_velocity']
n = inputs['normal_vectors']
A = inputs['surface_areas']
l = inputs['bound_vortices']
ic_tot = inputs['influence_coefficients_total']
v_inf = freestream_velocity
v_inf_vec = v_inf*freestream_unit_vector
lin_numerator = np.pi * v_inf * A * np.sum(n * v_inf_vec, axis=1)
lin_denominator = (np.linalg.norm(np.cross(v_inf_vec, l), axis=1) - np.pi * v_inf * A * np.sum(np.sum(n * ic_tot, axis=2), axis=1))
lin_vtx_str = lin_numerator / lin_denominator
outputs['vortex_strengths'] = lin_vtx_str
However, when the problem is run for the first time, any inputs not explicitly set with p.set_val() are all 1s. This causes guess_nonlinear() to give a bad output and so the system fails to converge:
As far as I can tell, the execution order for the LLT group is correct, and the geometry components should be being executed before the implicit component. I'm confused as to why this doesn't seem to actually be happening when the code is run, and instead these inputs are taking their default values.
What do I need to change to get this to work properly? Additionally, I've found difficulty in getting LNBGS to converge (hence adding guess_nonlinear()) during optimisation - only DirectSolver gets all the way through the optimisation without issues, but it's very slow for large numbers of LLT nodes). How can I improve the linear and nonlinear solver selection, and improve the reliability of the iterative solver?
Note: Thanks for providing a testable example. It made figuring out the answer to your question a lot simpler. Your problem was a bit subtle and I would not have been able to give a good answer without runnable code
Your first question: "Why are all the inputs 1"
"Short" Answer
You have put the nonlinear solver to high in the model hierarchy, which then included a key precurser component that computed your input values. By moving the solver down to a lower level of the model, I was able to ensure that the precurser component (LTTGeometry) ran and had valid outputs before you got to the guess_nonlinear of implicit component.
Here is what you had (Notice the implicit solver included LTTGeometry even though the data cycle does not require that component:
I moved both the nonlinear solver and the linear solver down into the LTTCycle group, which then allows the LTTGeometry component to execute before getting to the nonlinear solver and guess_nonlinear step:
My fix is only partially correct, since there is a secondary cycle from the TestCL component that also needs a solver and does not have one. However, that cycle still does not involve the LTTGeometry group. So the fully correct fix is to restructure you model top run geometry first, and then put the LTTCycle and TestCL groups together so you can run a solver over just them. That was a bit more hacking than I wanted to do on your test problem, but you can see the general idea from the adjusted N2 above.
Long Answer
The guess_nonlinear sequence in OpenMDAO does NOT run the compute method of explicit components or of groups. It follows the execution hierarchy, and calls any guess_nonlinear that it finds. So that means that any explicit components you have in your model will NOT get executed, their outputs will not get updated with computed values, and those computed values will not get passed to the inputs of downstream components.
Things get a little tricky when you have deep model hierarchies. The guess_nonlinear method is called as the first step in the nonlinear solver process. If you have a NonLinearRunOnce solver at the top level, it will follow the compute chain down the line calling compute or solve_nonlinear on each child and doing a data transfer after each one. If one of those children happens to be a group with a nonlinear solver, then that solver will call guess_nonlinear on its children (grandchildren of the top group with the NonLinearRunOnce solver) as the first step. So any outputs that were computed by the siblings of this group will be valid, but none of the outputs from the grandchild level will have been computed yet.
You may be wondering why not just have the guess_nonlinear method call the compute for any explicit components? There is a difficult to balance trade off here. If you assume that all explicit components are very cheap to run, then it might make sense to run the compute methods --- or it might not. A lot depends on the cyclic data structure. If some early component in the group needs guesses from the later one, then running its compute isn't going to help you much at all. Perhaps more importantly though, not all explicit components are cheap to run. You might have a very expensive computation, and calling compute as part of the guess process would be way too costly.
The compromise here, if you need some kind of top level guess process, is that you can implement guess_nonlinear at the group level. It's less common to do, but it gives you total control over what happens. You can call whatever you need to call in whatever sequence.
So the absolute key thing to remember is that the only data you have available to you when a guess_nonlinear is called is any data that was computed before your containing solver was executed. That means any thing that was computed before you got to the model scope of the containing solver (not the scope of the component with the guess_method itself).
Your second question: "How can I speed this up when the number of nodes gets large?"
This one not possible to give a generic answer to at all. I noticed that you have already specified sparse partial derivatives. That is a great start, but if its still not fast enough for you then it means you're reaching the limits of what you can do with a DirectSolver. You note that this solver is the only one that gets you through the optimization without issues, which I will take to mean that ScipyKryloventer link description here and PetscKrylov are not converging the linear system well for you --- at least not by themselves. Thats not surprising, as krylov solvers almost always require some kind of preconditioner... and this is why I can't offer a generic answer. Setting up efficient linear solvers for larger-scale compute is a tricky subject. If you look into the literature, you'll find some good suggestions. You can also study open source implementations like VSPAero for some tips.
effectively, you've reached the limit of what simple linear solvers can offer you. From this point forward, OpenMDAO can help a bit by making it easier to implement some preconditioning, but you'll have to suffer the math side yourself.
Currently I have a system that measures the GPS coordinates of an object. The object is first detected and then using trigonometry, the GPS coordinates are determined, as we know of the GPS coordinates of the camera itself.
However, the camera is on a moving object and therefore the data for the GPS coordinates can be quite noisy. In order to tackle this, I have decided to use an EKF in accordance with the system at hand.
In order to integrate this system into the program that I am currently using, I have decided to use this library here: https://github.com/simondlevy/TinyEKF. However, here it says that
TinyEKF requires you to write only a single model function, filling in
the values of the state-transition function f(x), its Jacobian matrix
F(x), the sensor function h(x), and its Jacobian H(x). The prediction
and update then handled automatically by passing the observation
vector z to the step function.
The output of my GPS code, is always [lat, lon]. What is x and what is z here? I don't understand how to create a state transition matrix. however, I am aware that a jacobian can be calculated from the state transition matrix, if I know what it is. Also, what is the sensor function(x)?
I'm doing a project with a lot of calculation and i got an idea is throw pieces of work to GPU, but i wonder whether could we retrieve results from GLSL, if it is posible, how?
GLSL does not provide outputs besides what is placed in the frame buffer.
To program a GPU and get results more conveniently, use CUDA (NVidia only) or OpenCL (cross-platform).
In general, what you want to do is use OpenCL for general-purpose GPU tasks. However, if you are insistent about pretending that OpenGL is not a rendering API...
Framebuffer Objects make it relatively easy to render to multiple outputs. This of course means that you have to structure your processing such that what gets rendered matches what you want. You can render to 32-bit floating-point "images", so you have access to plenty of precision. The biggest difficulty is what I stated: figuring out how to structure your task to match rendering.
It's a bit easier when using transform feedback. This is the ability to write the output of the vertex (or geometry) shader processing to a buffer object. This still requires structuring your tasks into something like rendering, but it's easier because vertex shaders have a strict one-vertex-to-one-vertex mapping. For every input vertex, there is exactly one output. And if you draw GL_POINTS, it's not too difficult to use attributes to pass the data that changes.
Both easier and harder is the use of shader_image_load_store. This is effectively the ability to read/write from/to arbitrary images "whenever you want". I put that last part in quotes because there are lots of esoteric rules about data race conditions: reading from a value written by another shader invocation and so forth. These are not trivial to deal with. You can try to structure your code to avoid them, by not writing to the same image location in the same shader. But in many cases, if you could do that, you could just render to the framebuffer.
Ultimately, it's pretty much impossible to answer this question in the general case, without knowing what exactly you're trying to actually do. How you approach GPGPU through a rendering API depends greatly on exactly what you're trying to compute.
I would like to build a Connect 4 engine which works using an artificial neural network - just because I'm fascinated by ANNs.
I'be created the following draft of the ANN structure. Would it work? And are these connections right (even the cross ones)?
Could you help me to draft up an UML class diagram for this ANN?
I want to give the board representation to the ANN as its input. And the output should be the move to chose.
The learning should later be done using reinforcement learning and the sigmoid function should be applied. The engine will play against human players. And depending on the result of the game, the weights should be adjusted then.
What I'm looking for ...
... is mainly coding issues. The more it goes away from abstract thinking to coding - the better it is.
The below is how I organized my design and code when I was messing with neural networks. The code here is (obviously) psuedocode and roughly follows Object Oriented conventions.
Starting from the bottom up, you'll have your neuron. Each neuron needs to be able to hold the weights it puts on the incoming connections, a buffer to hold the incoming connection data, and a list of its outgoing edges. Each neuron needs to be able to do three things:
A way to accept data from an incoming edge
A method of processing the input data and weights to formulate the value this neuron will be sending out
A way of sending out this neuron's value on the outgoing edges
Code-wise this translates to:
// Each neuron needs to keep track of this data
float in_data[]; // Values sent to this neuron
float weights[]; // The weights on each edge
float value; // The value this neuron will be sending out
Neuron out_edges[]; // Each Neuron that this neuron should send data to
// Each neuron should expose this functionality
void accept_data( float data ) {
in_data.append(data); // Add the data to the incoming data buffer
}
void process() {
value = /* result of combining weights and incoming data here */;
}
void send_value() {
foreach ( neuron in out_edges ) {
neuron.accept_data( value );
}
}
Next, I found it easiest if you make a Layer class which holds a list of neurons. (It's quite possible to skip over this class, and just have your NeuralNetwork hold a list of list of neurons. I found it to be easier organizationally and debugging-wise to have a Layer class.) Each layer should expose the ability to:
Cause each neuron to 'fire'
Return the raw array of neurons that this Layer wraps around. (This is useful when you need to do things like manually filling in input data in the first layer of a neural network.)
Code-wise this translates to:
//Each layer needs to keep track of this data.
Neuron[] neurons;
//Each layer should expose this functionality.
void fire() {
foreach ( neuron in neurons ) {
float value = neuron.process();
neuron.send_value( value );
}
}
Neuron[] get_neurons() {
return neurons;
}
Finally, you have a NeuralNetwork class that holds a list of layers, a way of setting up the first layer with initial data, a learning algorithm, and a way to run the whole neural network. In my implementation, I collected the final output data by adding a fourth layer consisting of a single fake neuron that simply buffered all of its incoming data and returned that.
// Each neural network needs to keep track of this data.
Layer[] layers;
// Each neural network should expose this functionality
void initialize( float[] input_data ) {
foreach ( neuron in layers[0].get_neurons() ) {
// do setup work here
}
}
void learn() {
foreach ( layer in layers ) {
foreach ( neuron in layer ) {
/* compare the neuron's computer value to the value it
* should have generated and adjust the weights accordingly
*/
}
}
}
void run() {
foreach (layer in layers) {
layer.fire();
}
}
I recommend starting with Backwards Propagation as your learning algorithm as it's supposedly the easiest to implement. When I was working on this, I had great difficulty trying to find a very simple explanation of the algorithm, but my notes list this site as being a good reference.
I hope that's enough to get you started!
There are a lot of different ways to implement neural networks that range from simple/easy-to-understand to highly-optimized. The Wikipedia article on backpropagation that you linked to has links to implementations in C++, C#, Java, etc. which could serve as good references, if you're interested in seeing how other people have done it.
One simple architecture would model both nodes and connections as separate entities; nodes would have possible incoming and outgoing connections to other nodes as well as activation levels and error values, whereas connections would have weight values.
Alternatively, there are more efficient ways to represent those nodes and connections -- as arrays of floating point values organized by layer, for example. This makes things a bit trickier to code, but avoids creating so many objects and pointers to objects.
One note: often people will include a bias node -- in addition to the normal input nodes -- that provides a constant value to every hidden and output node.
I've implemented neural networks before, and see a few problems with your proposed architecture:
A typical multi-layer network has connections from every input node to every hidden node, and from every hidden node to every output node. This allows information from all of the inputs to be combined and contribute to each output. If you dedicate 4 hidden nodes to each input then you will losing some of the network's power to identify relationships between the inputs and outputs.
How will you come up with values to train the network? Your network creates a mapping between board positions and the optimal next move, so you need a set of training examples that provide this. End game moves are easy to identify, but how do you tell that a mid-game move is "optimal"? (Reinforcement learning can help out here)
One last suggestion is to use bipolar inputs (-1 for false, +1 for true) since this can speed up learning. And Nate Kohl makes a good point: every hidden & output node will benefit from having a bias connection (think of it as another input node with a fixed value of "1").
Your design will be highly dependant on the specific type of reinforcment learning that you plan to use.
The simplest solution would be to use back propogation. This is done by feeding the error back into the network (in reverse fashion) and using the inverse of the (sigmoid) function to determine the adjustment to each weight. After a number of iterations, the weights will automatically get adjusted to fit the input.
Genetic Algorithms are an alternative to back-propogation which yield better results (although a bit slower). This is done by treating the weights as a schema that can easily be inserted and removed. The schema is replaced with a mutated version (using principles of natural selection) several times until a fit is found.
As you can see, the implementation for each of these would be drastically different. You could try to make the network generic enough to adapt to each type of implementation but that may overcomplicate it. Once you are in production, you will usually only have one form of training (or ideally your network would already be trainined).
The sensor module in my project consists of a rotating camera, that collects noisy information about moving objects in the surrounding environment.
The information consists of distance, angle and relative change of the moving objects..
The limiting view range of the camera makes it essential to rotate the camera periodically to update environment information...
I was looking for algorithms / ways to model these information, in order to be able to guess / predict / learn motion properties of these object..
My current proposed idea is to store last n snapshots of each object in a queue. I take weighted average of positions and velocities of moving object, but I think it is a poor method...
Can you state some titles that suit this case?
Thanks
Kalman {Extended, unscented, ... } filters and particle filters only after reading about Kalman filters.
Kalman filters learn and predict the correct data from noisy data with a Gaussian assumption, so it may be of use to you. If you need non-Gaussian methods, look at the particle filter.