I am designing a neural network to classify a class imbalanced sparse features data. It’s a binary classification problem. I used SMOTE to resolve the class imbalance issues. How to deal with the sparse features with neural network.
My task for my university assignment is to create AI for a "MOBA" style strategy game. I have looked into using neural networks for this. I cannot see any need to train the network beforehand.
In other words, would it still be considered as a neural network if I hard code in the weights and simply apply minor weight changes at runtime?
Neural network is one thing (a structure of neurons, synapses, etc.) and the learning algorithm is a completely different thing.
So if you ask whether a NN without learning algorithm can still be called NN I think the answer is yes, it can.
I am wondering what type of machine learning it uses and if someone could explain it to me. I have researched the different types and am unable to disngtuish what type it is due to my lack of knowledge.
While it's hard to say for sure, the fact that they're using deep learning on GPUs - pointing towards neural networks [1] seems to suggest that they're using a combination of unsupervised and supervised learning. The latter for bootstrapping the bot, and the former to learn on the job.
[1] http://www.existor.com/ai-parallel
call it weakly-supervised learning since we do not have exact labels but have intended document types. I dont know exaclty Cleverbot but state of the art takes large amount of documents and find models sequence relations of words by using Recurrent Neural Networks and LSTM in particular. Sometimes before it was Hidden Markov Models but now Deep Learning changed the game. If you search NLP with Recurrent Neural Network Google blows information.
Is a neural network a lazy or eager learning method? Different web pages say different things so I want to get a solid answer with good literature to back it up. The most obvious book to look in would be Mitchell's famous Machine Learning book but skimming through the whole thing I can't see the answer. Thanks :).
Looking at the definition of the terms lazy and eager learning, and knowing how a neural network works, I believe that it is clear that it is eager. A trained network is a generalisation function, all the weights and paths used to arrive at a classification are entirely determined by training data, but the training data itself is not retained for the purposes of the decision making.
An important distinction is that a Lazy system stores its training data and uses it directly to determine a solution. An eager system determines a function from the training data, and thereafter the training data is no longer required. That is to say you cannot determine what the training data was from an eager system's function. A neural network certainly fits that description. An eager system can therfore be very storage efficient, but conversely is non-deterministic, in the sense that it is not possible to determine how or why it arrived a a particular solution, so problems of poor or inappropriate training data may be difficult deal with.
The eager article linked above even gives artificial neural networks as an example. You might of course prefer a cited text to Wikipedia but the page has existed with that assertion since 2007 without contradictory edits, so I'd say that was pretty robust.
Some neural networks are eager learners, and some are lazy. Feedforward neural networks (as are commonly trained by some variant of backpropagation) are eager: they attempt to derive a representation of the underlying relationships in the data at the time of training. Radial basis function networks (such as probabilistic NN or generalized regression NN), on the other hand, are lazy learners (very much like k-nearest neighbors, the classic lazy learner).
A neural network is generally considered to be an "eager" learning method.
"Eager" learning methods are models that learn from the training data in real-time, adjusting the model parameters as new examples are presented. Neural networks are an example of an eager learning method because the model parameters are updated during the training process, as the algorithm iteratively processes the training examples. This allows the model to adapt and improve its performance as more examples are seen.
On the other hand, "lazy" learning methods, also known as instance-based or memory-based learning, only learn from the training data when a new example is presented. The model does not update its parameters during the training process but instead, it memorizes the training data and uses it to make predictions. Lazy learning methods typically require less computation time to make predictions than eager learning methods, but they may not perform as well on unseen data.
In general, neural networks are considered eager learning methods because their parameters are updated during the training process.
Here are a few literature references:
"Eager Learning vs. Lazy Learning" by R. S. Michalski, J. G. Carbonell, and T. M. Mitchell. This paper provides a comprehensive overview of the distinction between eager and lazy learning, and discusses the strengths and weaknesses of each approach. It was published in Machine Learning, 1983.
"An overview of instance-based learning algorithms" by A. K. Jain and R. C. Dubes. This book chapter provides an overview of the main concepts and techniques used in instance-based or lazy learning, and compares them to other types of learning algorithms, such as decision trees and neural networks. It was published in "Algorithms for Clustering Data" by Prentice-Hall, Inc. in 1988.
" Machine Learning" by Tom Mitchell. This book provides a comprehensive introduction to the field of machine learning, including the concepts of eager and lazy learning. It covers a wide range of topics, from supervised and unsupervised learning to deep learning and reinforcement learning. It was published by McGraw-Hill Education in 1997.
"Introduction to Machine Learning" by Alpaydin, E. This book provides an introduction to the field of machine learning, including the concepts of eager and lazy learning, as well as a broad range of machine learning algorithms. It was published by MIT press in 2010
It's also worth noting, that this classification of lazy and eager learning is not always clear cut and can be somewhat subjective, and some algorithms can belong to both categories, depending on the specific implementation.
I want to use Bayesian Network mechanism for spam filtering. How do you think it should look a proper topology of the network? What about naive Bayes model? (The naive Bayes model is sometimes called a Bayesian classifier)
If you're new to spam filtering, it'd be a good idea to start with something simple like a naive Bayesian classifier. That way you get familiar with the issues involved in handling the data (reading the email, classifying it, storing your lexicon, etc.) without getting too bogged down in the actually classification code. Once you have the basics of your program working, you can go on to more advanced types of filtering.
I found the discussion in the book Ending Spam to be quite helpful.