Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 3 years ago.
Improve this question
I have written an artifical neural network (ANN) implementation for myself (it was fun). I am thinking now about where can I use it.
What are the key areas in the real world, where ANN is being used?
ANNs are an example of a "learning" system, one that "trains" on input data (in some domain) in order to effectively classify (unseen) data in that domain. They've been used for everything from character recognition to computer games and beyond.
If you're trying to find a domain, pick some topic or field that interests you, and see what kinds of classification problems exist there.
Most often for classifying noisy inputs into fixed categories, like handwritten letters into their equivalent character, spoken voice into phonemes, or noisy sensor readings into a set of fixed values. Usually, the set of categories is small (23 letters, couple of dozen phonemes, etc.)
Others will point out how all these things are better done with specialized algorithms....
I once wrote an ANN to predict the stock market. It succeeded with about 80% accuracy.
The cue here was to first get hold of a couple of million rows of real stock data. I used this data to train the network and prime it for real data. There were about 8-10 input variables and a single output value that would indicate the predicted value of the stock on the next day.
You could also check out the (ancient) ALVINN network where a car learnt to drive by itself by observing road data when a human driver was behind the wheel.
ANNs are also widely used in bioinformatics.
Related
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 12 months ago.
Improve this question
There are some blog-articles in project I work, and,
I guess, its text field should be limited somehow
(probably it would be a JSON field one day)
There are no limitations in sense of domain -
user can write as much as he wants, but,
just to prevent DB harm by hack-attacks with uncommonly huge amounts of text, guess, some limit is needed.
As SO Q/A says:
PostgreSQL limit text field with 1GB Is there a maximum length when storing into PostgreSQL TEXT
http POST limits depend on browser (2GB - 4GB) https://serverfault.com/questions/151090/
By rumors, Nginx's default client_max_body_size is 1MB
So, how to deal with all of that?
Probably, there are some practice like:
"Just limit it with million chars in app-lvl and don't worry"?
This is an interesting question. If I understand correctly you are working on an application where the DB entry will essentially be a blog post (typically 500-1000 words based on most blogs I read).
You are storing the blog post as a text field in your database. You are quite reasonably worried about what happens with large blobs of data.
I fully advocate for you having a limit on the amount of data the user can enter. Without fully understanding the architecture of your system it's impossible to say what is the theoretical max size based on the technologies used.
However, it is better to look at this from a user perspective. What is the maximum reasonable amount of text for you to have to store, then let's add a bit more say 10% because let's face it users will do the unexpected. You can then add an error condition when someone tried to enter more data.
My reason for proposing this approach is a simple one, once you have defined a maximum post size you can use boundary value analysis (testing just either side of the limit) to prove that your product behaves correctly both just below and at the limit. This way you will know and can explain the product behaviour to the users etc.
If you choose to let the architecture define the limit then you will have undefined behaviour. You will need to analyze each component in turn to work out the maximum size they will accept and the behaviour they exhibit when that size is exceeded.
Typically (in my experience) developers don't put this effort in and let the users do the testing for them. This of course is generally worse because the user reports a strange error message and then the debugging is ultimately time consuming and expensive.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 4 years ago.
Improve this question
Just out of curiosity because I've always wondered this. How does the application Shazam work? I know how to use it, I'm speaking in terms of programming. How does the application listen to any part of a song and then give you the results? Obviously it receives it's song information from a database, but there is NO way someone could enter every single song known to man in that database. Also, how does Shazam not constantly update all the time? New songs are constantly being released yet it was like Shazam already had the future songs programmed into it. This is just mind boggling to me, and I would just like to know how exactly this all works. I know this is not a help question, but could someone please clarify? Thanks!
Shazam only starts with Fourier transforms (which isn't surprising since pretty much all audio processing works this way).
You can read Avery Wang's original paper, if you like. He is the inventor of the Shazam algorithm. I happen to think that it is best explained as a nearest neighbor technique, which is why we included it as an example in Chapter 9 of "Data Mining Techniques, 3rd Edition".
You might be interested in what we have to say there (http://www.amazon.com/Data-Mining-Techniques-Relationship-Management/dp/0470650931/ref=pd_sim_b_5).
They don't say much on the link diciu posted.
The algorithm is based on Fourier's waves, which allows expressing a mathematical function as a linear sum of harmonic functions. This transform allows mapping between time to frequency which is exactly what you need in order to create voice recognition. I find it hard to believe that Shazaam has a patent over Fourier's transformation. But if you try to build a "2nd Shazaam" you'll probably fail since they already took over all the market...
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 4 years ago.
Improve this question
What does number of hidden layers in a multilayer perceptron neural network do to the way neural network behaves? Same question for number of nodes in hidden layers?
Let's say I want to use a neural network for hand written character recognition. In this case I put pixel colour intensity values as input nodes, and character classes as output nodes.
How would I choose number of hidden layers and nodes to solve such problem?
Note: this answer was correct at the time it was made, but has since become outdated.
It is rare to have more than two hidden layers in a neural network. The number of layers will usually not be a parameter of your network you will worry much about.
Although multi-layer neural networks with many layers can represent
deep circuits, training deep networks has always been seen as somewhat
of a challenge. Until very recently, empirical studies often found
that deep networks generally performed no better, and often worse,
than neural networks with one or two hidden layers.
Bengio, Y. & LeCun, Y., 2007. Scaling learning algorithms towards AI. Large-Scale Kernel Machines, (1), pp.1-41.
The cited paper is a good reference for learning about the effect of network depth, recent progress in teaching deep networks, and deep learning in general.
The general answer is to for picking hyperparameters is to cross-validate. Hold out some data, train the networks with different configurations, and use the one that performs best on the held out set.
Most of the problems I have seen were solved with 1-2 hidden layers. It is proven that MLPs with only one hidden layer are universal function approximators (Hornik et. al.). More hidden layers can make the problem easier or harder. You usually have to try different topologies. I heard that you cannot add an arbitrary number of hidden layers if you want to train your MLP with backprop because the gradient will become too small in the first layers (I have no reference for that). But there are some applications where people used up to nine layers. Maybe you are interested in a standard benchmark problem which is solved by different classifiers and MLP topologies.
Besides the fact that cross-validation on different model configurations(no. of hidden layers OR neurons per layer) will lead you to choose better configuration.
One approach is training a model, as big and deep as possible and use dropout regularization to turn off some neurons and reduce overfitting.
the reference to this approach can be seen in this paper.
https://www.cs.toronto.edu/~hinton/absps/JMLRdropout.pdf
All the above answers are of course correct but just to add some more ideas:
Some general rules are the following based on this paper: 'Approximating Number of Hidden layer neurons in Multiple Hidden Layer BPNN Architecture' by Saurabh Karsoliya
In general:
The number of hidden layer neurons are 2/3 (or 70% to 90%) of the size of the input layer. If this is insufficient then number of output layer neurons can be added later on.
The number of hidden layer neurons should be less than twice of the number of neurons in input layer.
The size of the hidden layer neurons is between the input layer size and the output layer size.
Keep always in mind that you need to explore and try a lot of different combinations. Also, using GridSearch you could find the "best model and parameters".
E.g. we can do a GridSearch in order to determine the "best" size of the hidden layer.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 7 years ago.
Improve this question
I am looking for a graduation project idea in AI and machine learning field...
The idea may require front-end user interface to attract users...
I am thinking of how AI and machine learning can help you in daily life..?
Any help/hint about new interesting ideas ?
Thanks
Edit:
I am talking about practical ideas that may be used in real life... Not an idea to prove theoretical things... Something like a OS (or an add on in existing one) that adapt with your way of work... or a word processor that helps you collecting information about what you are writing..
What about an project that uses Markov-Chain text generation to generate answers for Stack Overflow questions? (^__^)
Two ideas:
Write a current, robust version of SHRDLU with understandable source code.
Write a SHRDLU-like program that manipulates actual code instead of imaginary blocks. Such a tool could be used for manipulating extremely large, complicated programs, including its own code!
Imagine giving commands like the following...
(a) Scan web site X and list any sentences you failed to parse.
(b) Scan document Y and list any grammar rules you didn't need.
(c) Instead of iterating over every element of "proplist" in your "search" function, only process the cdr of "proplist" if the initial call to "lookup" returns nil. After you make the modification, confirm the sentence "pick up a very very big block" will succeed and the sentence "pick up a very and very big block" will fail.
(d) Your "conjoin" grammar currently requires a coordinator word like "and", but that requirement is wrong. Split your "coordination" grammar into "syndetic coordination" and "asyndetic coordination" as follows: conjoins using "and", as in "quickly and quietly, he walked into the bank" are called "syndetic coordinations". Conjoins without a coordinator, as in "quickly, quietly, he walked into the bank" are "asyndetic coordinations". Now scan corpus Z to see if fewer sentences fail to parse.
One component of intelligence is imagination.
It wouldn't take much to Google for "artificial intelligence research projects" and see what other people are doing at other schools. Since it's not a Ph.D., there's no uniqueness requirement for you.
You could also look at Peter Norvig's text to see what's been done before and adapt it.
I'd also recommend doing something with the reams of data that's available to you on the web. Try thinking about "Programming Collective Intelligence" and "Beautiful Data" to see how you could use information to teach a program how to adapt its behavior based on new information (neural nets, genetic algorithms, ant colony algorithms, etc.)
What interests you?
AI is used in a great many areas, so find something you are passionate about and then see how to use AI for it.
For example, if you are interested in games, then you could find an interesting algorithm for the ghosts in Pac-man to chase, and use some more interesting mazes. You may find someone that is interested in doing a 3D project and they could write a 3D version and your algorithm could be more interesting.
Or, you may be interested in robotics. Again, it would be ideal if you could find someone with an interest in making a robot and you could write the AI part. So, for example, you could see if you can figure out how to get a robot to determine the difference between a farm crop and a weed/grass.
Basically, your starting point should be on what really interests you.
Perform a clustering on documents, as done by ex-clusty search engine. This clearly is an attractive application.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 7 years ago.
Improve this question
How would one design a neural network for the purpose of a recommendation engine. I assume each user would require their own network, but how would you design the inputs and the outputs for recommending an item in a database. Are there any good tutorials or something?
Edit: I was more thinking how one would design a network. As in how many input neurons and how the output neurons point to a record in a database. Would you have say 6 output neurons, convert it to an integer (which would be anything from 0 - 63) and that is the ID of the record in the database? Is that how people do it?
I would suggest looking into neural networks using unsupervised learning such as self organising maps. It's very difficult to use normal supervised neural networks to do what you want unless you can classify the data very precisely for learning. self organising maps don't have this problem because the network learns the classification groups all on their own.
have a look at this paper which describes a music recommendation system for music
http://www.springerlink.com/content/xhcyn5rj35cvncvf/
and many more papers written about the topic from google scholar
http://www.google.com.au/search?q=%09+A+Self-Organizing+Map+Based+Knowledge+Discovery+for+Music+Recommendation+Systems+&ie=utf-8&oe=utf-8&aq=t&rls=com.ubuntu:en-US:official&client=firefox-a&safe=active
First you have to decide what exactly you are recommending and under what circumstances. There are many things to take into account. Are you going to consider the "other users who bought X also bought Y?" Are you going to only recommend items that have a similar nature to each other? Are you recommending items that have a this-one-is-more-useful-with-that-one type of relationship?
I'm sure there are many more decisions, and each one of them has their own goals in mind. It would be very difficult to train one giant network to handle all of the above.
Neural networks all boil down to the same thing. You have a given set of inputs. You have a network topology. You have an activation function. You have weights on the nodes' inputs. You have outputs, and you have a means to measure and correct error. Each type of neural network might have its own way of doing each of those things, but they are present all the time (to my limited knowledge). Then, you train the network by feeding in a series of input sets that have known output results. You run this training set as much as you'd like without over or under training (which is as much your guess as it is the next guy's), and then you're ready to roll.
Essentially, your input set can be described as a certain set of qualities that you believe have relevance to the underlying function at hand (for instance: precipitation, humidity, temperature, illness, age, location, cost, skill, time of day, day of week, work status, and gender may all have an important role in deciding whether or not person will go golfing on a given day). You must therefore decide what exactly you are trying to recommend and under what conditions. Your network inputs can be boolean in nature (0.0 being false and 1.0 being true, for instance) or mapped in a pseudo-continuous space (where 0.0 may mean not at all, .45 means somewhat, .8 means likely, and 1.0 means yes). This second option may give you the tools to map confidence level for a certain input, or simple a math calculation you believe is relevant.
Hope this helped. You didn't give much to go on :)