Connect 4 with neural network: evaluation of draft + further steps - artificial-intelligence

Question

I would like to build a Connect 4 engine which works using an artificial neural network - just because I'm fascinated by ANNs.
I'be created the following draft of the ANN structure. Would it work? And are these connections right (even the cross ones)?
Could you help me to draft up an UML class diagram for this ANN?
I want to give the board representation to the ANN as its input. And the output should be the move to chose.
The learning should later be done using reinforcement learning and the sigmoid function should be applied. The engine will play against human players. And depending on the result of the game, the weights should be adjusted then.
What I'm looking for ...
... is mainly coding issues. The more it goes away from abstract thinking to coding - the better it is.

Answer 1

The below is how I organized my design and code when I was messing with neural networks. The code here is (obviously) psuedocode and roughly follows Object Oriented conventions.
Starting from the bottom up, you'll have your neuron. Each neuron needs to be able to hold the weights it puts on the incoming connections, a buffer to hold the incoming connection data, and a list of its outgoing edges. Each neuron needs to be able to do three things:
A way to accept data from an incoming edge
A method of processing the input data and weights to formulate the value this neuron will be sending out
A way of sending out this neuron's value on the outgoing edges
Code-wise this translates to:
// Each neuron needs to keep track of this data
float in_data[]; // Values sent to this neuron
float weights[]; // The weights on each edge
float value; // The value this neuron will be sending out
Neuron out_edges[]; // Each Neuron that this neuron should send data to
// Each neuron should expose this functionality
void accept_data( float data ) {
in_data.append(data); // Add the data to the incoming data buffer
}
void process() {
value = /* result of combining weights and incoming data here */;
}
void send_value() {
foreach ( neuron in out_edges ) {
neuron.accept_data( value );
}
}
Next, I found it easiest if you make a Layer class which holds a list of neurons. (It's quite possible to skip over this class, and just have your NeuralNetwork hold a list of list of neurons. I found it to be easier organizationally and debugging-wise to have a Layer class.) Each layer should expose the ability to:
Cause each neuron to 'fire'
Return the raw array of neurons that this Layer wraps around. (This is useful when you need to do things like manually filling in input data in the first layer of a neural network.)
Code-wise this translates to:
//Each layer needs to keep track of this data.
Neuron[] neurons;
//Each layer should expose this functionality.
void fire() {
foreach ( neuron in neurons ) {
float value = neuron.process();
neuron.send_value( value );
}
}
Neuron[] get_neurons() {
return neurons;
}
Finally, you have a NeuralNetwork class that holds a list of layers, a way of setting up the first layer with initial data, a learning algorithm, and a way to run the whole neural network. In my implementation, I collected the final output data by adding a fourth layer consisting of a single fake neuron that simply buffered all of its incoming data and returned that.
// Each neural network needs to keep track of this data.
Layer[] layers;
// Each neural network should expose this functionality
void initialize( float[] input_data ) {
foreach ( neuron in layers[0].get_neurons() ) {
// do setup work here
}
}
void learn() {
foreach ( layer in layers ) {
foreach ( neuron in layer ) {
/* compare the neuron's computer value to the value it
* should have generated and adjust the weights accordingly
*/
}
}
}
void run() {
foreach (layer in layers) {
layer.fire();
}
}
I recommend starting with Backwards Propagation as your learning algorithm as it's supposedly the easiest to implement. When I was working on this, I had great difficulty trying to find a very simple explanation of the algorithm, but my notes list this site as being a good reference.
I hope that's enough to get you started!

Answer 2

There are a lot of different ways to implement neural networks that range from simple/easy-to-understand to highly-optimized. The Wikipedia article on backpropagation that you linked to has links to implementations in C++, C#, Java, etc. which could serve as good references, if you're interested in seeing how other people have done it.
One simple architecture would model both nodes and connections as separate entities; nodes would have possible incoming and outgoing connections to other nodes as well as activation levels and error values, whereas connections would have weight values.
Alternatively, there are more efficient ways to represent those nodes and connections -- as arrays of floating point values organized by layer, for example. This makes things a bit trickier to code, but avoids creating so many objects and pointers to objects.
One note: often people will include a bias node -- in addition to the normal input nodes -- that provides a constant value to every hidden and output node.

Answer 3

I've implemented neural networks before, and see a few problems with your proposed architecture:
A typical multi-layer network has connections from every input node to every hidden node, and from every hidden node to every output node. This allows information from all of the inputs to be combined and contribute to each output. If you dedicate 4 hidden nodes to each input then you will losing some of the network's power to identify relationships between the inputs and outputs.
How will you come up with values to train the network? Your network creates a mapping between board positions and the optimal next move, so you need a set of training examples that provide this. End game moves are easy to identify, but how do you tell that a mid-game move is "optimal"? (Reinforcement learning can help out here)
One last suggestion is to use bipolar inputs (-1 for false, +1 for true) since this can speed up learning. And Nate Kohl makes a good point: every hidden & output node will benefit from having a bias connection (think of it as another input node with a fixed value of "1").

Answer 4

Your design will be highly dependant on the specific type of reinforcment learning that you plan to use.
The simplest solution would be to use back propogation. This is done by feeding the error back into the network (in reverse fashion) and using the inverse of the (sigmoid) function to determine the adjustment to each weight. After a number of iterations, the weights will automatically get adjusted to fit the input.
Genetic Algorithms are an alternative to back-propogation which yield better results (although a bit slower). This is done by treating the weights as a schema that can easily be inserted and removed. The schema is replaced with a mutated version (using principles of natural selection) several times until a fit is found.
As you can see, the implementation for each of these would be drastically different. You could try to make the network generic enough to adapt to each type of implementation but that may overcomplicate it. Once you are in production, you will usually only have one form of training (or ideally your network would already be trainined).