Closed. This question is off-topic. It is not currently accepting answers.
Want to improve this question? Update the question so it's on-topic for Stack Overflow.
Closed 10 years ago.
Improve this question
I'm studying programming on my own and I would like to have an idea how to solve this problem.
I have been given the set of resistors with given resistances and a given value restot. I can pick a given number of those resistors. How can I make a circuit which resistance is as near as possible to restot? A programmer told me that that one can use genetic algorithms but I'm not limited to use such.
I guess I have to make a linear system of equations using Kirchoff's laws to make equations but as I don't have very much experience on electricity problems nor numerical algorithms to linear systems so I would like to have some guidance about how can I make those equations automatically to computers memory as the system changes all the time. And how can I make sure that the algorithm converges to a better solutions?
The problem is from a Finnish discussion forum.
Resistors can either exist in series or in parallel, and their resistances add up differently (add values for series, add reciprocals for parallel).
You can also have networks for resistors in series and parallel.
This sounds to me like a classic case of a recursive data structure, and you could probably represent it as a tree, in a similar way to a binary expression tree: http://en.wikipedia.org/wiki/Binary_expression_tree
Combine that some exploratory tree building (you should look into the way Prolog does this) and you can find the best combination of resistors that gets close to your total.
No genetic algorithms in this approach, although you could take a genetic approach to building and refining the tree.
To apply a genetic algorithm you would need to find a way to represent, mutate and combine the "DNA" of a resistor network.
One way would be to:
Add some number of 0 ohm resistors to your resister set (representing wires).
Number the resistors from 1 to N
For some M, imagine a set of M junctions including the source (1) and sink (M).
You could define which junctions the two endpoints of each resistor are connected to as the unique identifier of a network. This is just an N-tuple of integer pairs in the range 1..M. This tuple can be the "DNA".
Then:
Generate a bunch of random networks from random tuples.
Calculate each networks resistence
Discard some amount of the population farthest from the target resistence.
Combine random pairs of them to form new networks. (perhaps by randomly selecting each resistor endpoint from either parent A or parent B with 50% probability)
Randomly change a few endpoints (mutation).
Goto 2
Not sure if it would actually work exactly like this, but you get the general idea.
There is undoubtably a better non-genetic algorithm, but you specifically asked for a genetic one so there you go.
If you are not limited to genetic algorithm, then I think you can also solve this problem with help of linear programming. You can encode the problem as below and ask a solver to give the answer for you.
Required Resistance Of Circuit = x ohms
// We want to have total 33 resistors.
selected_in_series_1 + selected_in_series_2 +... + selected_in_series_211 + selected_in_parallel_1 + selected_in_parallel_2 + ... + selected_in_parallel_211 = 33
// Resistor in Series
(selected_in_series_1 * Resistor_1) + (selected_in_series_2 * Resistor_2) + ..(selected_in_series_211 * Resistor_211) = total_resistence_in series
// Similarly write formula for parallel
(selected_in_parallel_1 * 1/Resistor_1) + (selected_in_parallel_2 * 1/Resistor_2) + ..(selected_in_parallel_211 * 1/Resistor_211) = 1/total_resistence_in parallel
total_resistence_in series + total_resistence_in parallel = Required Resistance Of Circuit
Related
I have a simple task to classify people by their height and hair length to either MAN or WOMAN category using a neural network. Also teach it the pattern with some examples and then use it to classify on its own.
I have a basic understanding of neural networks but would really need some help here.
I know that each neuron divides the area to two subareas, basically that is why P = w0 + w1*x1 + w2*x2 + ... + wn*xn is being used here (weights are just moving the line if we consider geometric representation).
I do understand that each epoche should modify the weights to get closer to correct result, yet I have never program it and I am hopeless about how to start.
How should I proceed, meaning: How can I determine the threshold and how should I deal with the inputs?
It is not a homework rather than task for the ones who were interested. I am and I would like to understand it.
Looks like you are dealing with a simple Perceptron with a threshold activation function. Have a look at this question. Since you ARE using a bias neuron (w0), you would set the threshold to 0.
You then simply take the output of your network and compare it to 0, so you would e.g. output class 1 if x < 0 and class 2 if x > 0. You could model the case x=0 as "indistinct".
For learning the weights you need to apply the Delta Learning Rule which can be implemented very easily. But be careful: a perceptron with a simple threshold activation function can only be correct if your data are linearly separable. If you have more complex data you will need a Multilayer Perceptron and a nonlinear activation function like the Logistic Sigmoid Function.
Have a look at Geoffrey Hintons Coursera Course, Lecture 2 for details.
I've been working with machine learning lately (but I'm not an expert) but you should look at the Accord.NET framework. It contains all the common machine learning algorithme out of the box. So it's easy to take an existing samples and modify it instead of starting from scratch. Also, the developper of the framework is very helpful in the forum available on the same page.
With the available samples, you may also discover something better than neural network like the Kernel Support Vector Machine. If you stick to the neural network, have fun modifying all the different variables and by tryout and error you will understand how it work.
Have fun!
Since you said:
I know that each neuron divides the area to two subareas
&
weights are just moving the line if we consider geometric representation
I think you want to use perseptron or ADALINE neural networks. These neural networks can just classify linear separable patterns. since your input data is complicated, It's better to use a Multi layer Non-Linear Neural network. (my suggestion is a two layer neural network with tanh activation function) . For training these network you should use back propagation algorithm.
For answering to
how should I deal with the inputs?
I need to know more details about the inputs( Like: are they just height and hair length or there is more, what is their range and your resolution and etc.)
If you're dealing with just height and hair length I suggest that divide heights and length in some classes (for example 160cm-165cm, 165cm-170cm & etc.) and for each one of these classes set an On/Off input neuron. then put a hidden layer after all classes related to heights and another hidden layer after all classes related to hair length (tanh activation function). Number of neurons in these two hidden layer is determined based on number of training cases.
then take these two hidden layer output and send them to an aggregation layer with 1 output neuron.
Closed. This question is off-topic. It is not currently accepting answers.
Want to improve this question? Update the question so it's on-topic for Stack Overflow.
Closed 10 years ago.
Improve this question
There are some related questions that I've come across (like this, this, this, and this) but they all deal with fitting data to a known curve. Is there a way to fit given data to an unknown curve? By which I mean, given some data the algorithm will give me a fit which is one function or a sum of functions. I'm programming in C, but I'm at a complete loss on how to use the gsl package to do this. I'm open to using anything that can (ideally) be piped through C. But any help on what direction I should look will be greatly appreciated.
EDIT: This is basically experimental (physics) data that I've collected, so the data will have some trend modified by additive gaussian distributed noise. In general the trend will be non-linear, so I guess that a linear regression fitting method will be unsuitable. As for the ordering, the data is time-ordered, so the curve necessarily has to be fit in that order.
You might be looking for polynomial interpolation, in the field of numerical analysis.
In polynomial interpolation - given a set of points (x,y) - you are trying to find the best polynom that fits these points. One way to do it is using Newton interpolation, which is fairly easy to program.
The field of numerical analysis and interpolations in specifics is widely studied, and you might be able to get some nice upper bound to the error of the polynom.
Note however, because you are looking for a polynom that best fits your data, and the function is not really a polynom - the scale of the error when getting far from your initial training set blasts off.
Also note, your data set is finite, and there are inifnite number (actually, non-enumerable infinity) of functions that can fit the data (exactly or approximately) - so which one out of these is the best might be specific to what you actually are trying to achieve.
If you are looking for a model to fit your data, note that linear regression and polynomial interpolations are at the opposite ends of the scale: polynomial interpolation might be an overfitting to a model, while a linear regression might be underfitting it, what exactly should be used is case specific and varies from one application to the other.
Simple polynomial interpolation example:
Let's say we have (0,1),(1,2),(3,10) as our data.
The table1 we get using newton method is:
0 | 1 | |
1 | 2 | (2-1)/(1-0)=1 |
3 | 9 | (10-2)/(3-1)=4 | (4-1)/(3-0)=1
Now, the polynom we get is the "diagonal" that ends with the last element:
1 + 1*(x-0) + 1*(x-0)(x-1) = 1 + x + x^2 - x = x^2 +1
(and that is a perfect fit indeed to the data we used)
(1) The table is recursively created: The first 2 columns are the x,y values - and each next column is based on the prior one. It is really easy to implement once you get it, the full explanation is in the wikipedia page for newton interpolation.
Another alternative is using linear regression, but multi-dimensional.
The trick here is to artificially generate extra dimensions. You can do so by simply implying some functions on the original data set. A common usage is doing it to generate polynoms to match the data, so in here the function you imply is f(x) = x^i for all i < k (where k is the degree of the polynom you want to get).
For example, the data set (0,2),(2,3) with k = 3 you will get extra 2 dimnsions, and your data set will be: (0,2,4,8),(2,3,9,27).
The linear-regression algorithm will find the values a_0,a_1,...,a_k for the polynom p(x) = a_0 + a_1*x + ... + a_k * x^k that minimized the error for each point in the data comparing to the predicted model (the value of p(x)).
Now, the problem is - when you start increasing the dimension - you are moving from underfitting (of 1 dimensional linear regression) to overfitting (when k==n, you effectively getting polynomial interpolation).
To "chose" what is the best k value - you can use cross-validation, and chose the k that minimized the error according to your cross-validation.
Note that this process can be fully automated, all you need is to iteratively check all k values in the desired range1, and chose the model with the k that minimized the error according to the cross-validation.
(1) The range could be [1,n] - though it will probably be way too time consuming, I'd go for [1,sqrt(n)] or even [1,log(n)] - but it is just a hunch.
You might want to use (Fast) Fourier Transforms to convert data to frequency domain.
With the result of the transform (a set of amplitudes and phases and frequencies) even the most twisted set of data can be represented by several functions (harmonics) of the form:
r * cos(f * t - p)
where r is the harmonic amplitude, f is the frequency an p the phase.
Finally, the unknonwn data curve is the sum of all harmonics.
I have done this in R (you have some examples of it) but I believe C has enough tools to manage it. It is also possible to pipe C and R but don't know much about it. This might be of help.
This method is really good for large chunks of data because it has complexities of:
1) decompose data with Fast Fourier Transforms (FTT) = O(n log n)
2) built the function with the resulting components = O(n)
I once wrote a Tetris AI that played Tetris quite well. The algorithm I used (described in this paper) is a two-step process.
In the first step, the programmer decides to track inputs that are "interesting" to the problem. In Tetris we might be interested in tracking how many gaps there are in a row because minimizing gaps could help place future pieces more easily. Another might be the average column height because it may be a bad idea to take risks if you're about to lose.
The second step is determining weights associated with each input. This is the part where I used a genetic algorithm. Any learning algorithm will do here, as long as the weights are adjusted over time based on the results. The idea is to let the computer decide how the input relates to the solution.
Using these inputs and their weights we can determine the value of taking any action. For example, if putting the straight line shape all the way in the right column will eliminate the gaps of 4 different rows, then this action could get a very high score if its weight is high. Likewise, laying it flat on top might actually cause gaps and so that action gets a low score.
I've always wondered if there's a way to apply a learning algorithm to the first step, where we find "interesting" potential inputs. It seems possible to write an algorithm where the computer first learns what inputs might be useful, then applies learning to weigh those inputs. Has anything been done like this before? Is it already being used in any AI applications?
In neural networks, you can select 'interesting' potential inputs by finding the ones that have the strongest correlation, positive or negative, with the classifications you're training for. I imagine you can do similarly in other contexts.
I think I might approach the problem you're describing by feeding more primitive data to a learning algorithm. For instance, a tetris game state may be described by the list of occupied cells. A string of bits describing this information would be a suitable input to that stage of the learning algorithm. actually training on that is still challenging; how do you know whether those are useful results. I suppose you could roll the whole algorithm into a single blob, where the algorithm is fed with the successive states of play and the output would just be the block placements, with higher scoring algorithms selected for future generations.
Another choice might be to use a large corpus of plays from other sources; such as recorded plays from human players or a hand-crafted ai, and select the algorithms who's outputs bear a strong correlation to some interesting fact or another from the future play, such as the score earned over the next 10 moves.
Yes, there is a way.
If you choose M selected features there are 2^M subsets, so there is a lot to look at.
I would to the following:
For each subset S
run your code to optimize the weights W
save S and the corresponding W
Then for each pair S-W, you can run G games for each pair and save the score L for each one. Now you have a table like this:
feature1 feature2 feature3 featureM subset_code game_number scoreL
1 0 1 1 S1 1 10500
1 0 1 1 S1 2 6230
...
0 1 1 0 S2 G + 1 30120
0 1 1 0 S2 G + 2 25900
Now you can run some component selection algorithm (PCA for example) and decide which features are worth to explain scoreL.
A tip: When running the code to optimize W, seed the random number generator, so that each different 'evolving brain' is tested against the same piece sequence.
I hope it helps in something!
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 3 years ago.
Improve this question
Can you tell me any ways to generate non-uniform random numbers?
I am using Java but the code examples can be in whatever you want.
One way is to create a skewed distribution by adding two uniform random numbers together (i.e. rolling 2 dice).
Try generating uniformly distributed random numbers, then applying your inverted non-uniform cumulative distribution function to each of them.
What distribution of deviates do you want?
Here is a technique which always works, but isn't always the most efficient. The cumulative distrubtion function P(x) gives the fraction of the time that values fall below x. Thus P(x)=0 at the lowest possible value of x and P(x)=1 at the highest possible value of x. Every distribution has a unique CDF, which encodes all the properties of the distrubtion in the way that P(x) rises from 0 to 1. If y is a uniform deviate on the interval [0,1], then x satisfying P(x)=y will be disributed according to your distribution. To make this work comuptationally, you just need a way computing the inverse of P(x) for your distribution.
The Meta.Numerics library defines a large number of commonly used distrubtions (e.g. normal, lognormal, exponential, chi squared, etc.) and has functions for computing the CDF (Distribution.LeftProbability) and the inverse CDF (Distribution.InverseLeftProbability) of each.
For specialized techniques that are fast for particular distrubtions, e.g. the Box-Muller technique for normaly distributed deviates, see the book Numerical Recipies.
If you are using Java then my Uncommons Maths library may be of interest. It includes classes for generating random numbers for Uniform, Gaussian, Poisson, Binomial and Exponential distributions. This article shows how you might use these distributions.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 4 years ago.
Improve this question
I want to play Tic-tac-toe using an artificial neural network. My configuration for the network is as follows:
For each of the 9 fields, I use 2 input neuron. So I have 18 input neurons, of course. For every field, I have 1 input neuron for a piece of Player 1 and 1 neuron for a piece of Player 2. In addition to that, I have 1 output neuron which gives an evaluation of the current board position. The higher the output value is, the better is the position for Player 1. The lower it is, the better is it for Player 2.
But my problem is: How could I code that neural network? My idea was to use an Array[1-18] for the input neurons. The values of this array are the input weights. The I would walk through the array using a loop. Whenever there is a neuron to be activated, I add the weight to the output value. So the output value is the sum of the weights of the activated input neurons:
Output = SUM(ActivatedInputNeurons)
Do you think this is a good way of programming the network? Do you have better ideas?
I hope you can help me. Thanks in advance!
Well, you have an input layer of 18 neurons, and an output layer of 1 neuron. That's OK. However, you need to give your neural net the opportunity to put the inputs into relation. For that, you need at least one intermediate layer. I would propose to use 9 neurons in the intermediate layer. Each of these should be connected to each input neuron, and the output neuron should be connected to each intermediate. Each such connection has a weight, and each neuron has an activation level.
Then, you go through all neurons, a layer at a time. The input layer is just activated with the board state. For all further neurons, you go through all its respective connections and sum over the product of the connected neuron's activation level and the weight of the connection. Finally, you calculate the activation level by applying a sigmoid function on this sum.
This is the working principle. Now, you need to train this net to get better results. There are several algorithms for this, you will have to do some googling and reading. Finally, you might want to adjust the number of neurons and layers when the results don't get convincing fast enough. For example, you could reduce the input layer to 9 neurons and activate them with +1 for an X and -1 for an O. Perhaps adding another intermediate layer yields better results, or increasing the number of neurons of a layer.
I don't particularly understand how you expect to get a meaningful summary of the board situation out of one output neuron. I would more look at having:
I I I O O O
I I I x O O O
I I I O O O
9 input neurons 9 output neurons
in a fully connected network, i.e. 81 weights. Then train the output neurons for the relative desirability of playing in that position.
Have a look at my Tic project. I've solved this problem with both neural network and genetic algorithm. The source code is freely available.
http://www.roncemer.com/tic-tac-toe-an-experiment-in-machine-learning
I think you should implement a 'traditional' feed-forward ANN using transfer functions, as that allows you to train it using back-propagation. The code for these usually ends up being a few lines of code, something like this:
SetupInputs();
for (l = 1 .. layers.count)
for (i = 0 .. layers[l].count)
sum = 0
for (j = 0 .. layers[l-1].count)
sum += layers[l-1][j] * weights[l-1][j]
layers[l][i] = TransferFunction(sum)
This is an excellent starter project for AI coding, but coming up with a complete solution will be way to big of an answer for SO.
As with most software, I recommend using an object-oriented design. For example: Define a Neuron class which has inputs, weights, and an output function. Then, create several of these Neuron objects in order to build your network.
See the wikipedia article on artificial neural networks for a good starting point.
Good luck with the code! Sounds like a lot of fun.
It is not a direct answer to your question, but you should have a look at the following framework/tool: SNNS or its Java counterpart JavaNNS. I'm pretty sure that there you'll find an answer to your question.
After adding the weights, you need to normalize the sum using a function, people usually use TANH, if you want to allow negative numbers.
edit:
Here is a java multilayer perceptron implementation that i worked on a few years ago. this one was used for checkers, but with less inputs you can use it for checkers too.
Also, you need to probably figure out a way to teach it to win, but thats another problem
You will save time if you use neural network library like FANN or Neuroph.
One way to encode your input is by 9 input neurons. The output is also good to be 9 neurons. What I do not see in the other replays is the size of the hidden layer. I suppose you are going to use MLP with traditional 3 layers. The size of the hidden layer is always mystery. I would try 10 hidden neurons.
If the transfer function is sigmoid you can encode input as follows:
0.0 - O player.
1.0 - X player.
0.5 - Empty.
The output of the ANN will be 9 real numbers. In this case some of the cells will be occupied already. You can search for the highest output value which corresponds to an empty cell.