Does anybody have any pointers to Naive Bayes Classifier Implementation preferably in C. I have 5 dimensional binary dataset. The class labels are also binary. I used Naive Bayes Classifier in Matlab with good results. However, is there any machine learning algorithm and its implementation which allows me to infer data from the class labels? Here in this case I want five dimensional binary data inferred from a binary class label. A sample of data is [1 1 0 1 0] and class is 0.
As you have a binary dataset, here is a nice implementation using C:
http://users.ics.tkk.fi/jhollmen/BernoulliMix/
It is a open source software that we are using currently in our course, you can actually check how he implemented the algorithm.
And about the question you made, here is my understanding.
What naive bayes classifier(NBC) does is to predict P(C|X) given some data and label. According to Bayes' theorem,
P(C|X) = \frac{P(X|C)P(C)}{P(X)}
which means that all you can do with predict the class of unknown data. Conversely, what you want to do there is P(X|C). Therefore, you can train your model like this,
P(X|C) = \frac{P(C|X)P(X)}{P(C)}
Accordingly, you have to assume distribution for your data...and stuff like that, therefore,it might be so accurate if you have a wrong assumption with your data. In you case, you have binary attributes X that is wanted be estimated from the label class, if you assume the attributes are independent, what you need to is like this,
P(C|X_1,X_2,X_3,X_4,X_5) \proportional P(X_1|C)P(X_2|C)P(X_3|C)P(X_4|C)P(X_5|C)P(C)
which is not so easy to solve.....
Hava a look at this package of the R-project:
http://www.stat.ucl.ac.be/ISdidactique/Rhelp/library/e1071/html/naiveBayes.html
http://cran.r-project.org/web/packages/e1071/index.html
You have tagged [C]: it is possible to link R with your own C-programs.
Related
I need to minimize a huge linear programming system where all related data (objective function, constraints) are stored in the memory in arrays and structures but not in lp file format or CPLEX
I saw that there are many solvers like here and here but the problem is how can I minimize the model without calling it from a file of a special format?
I did the same work previously in R and Python by solving the model directly after producing it without the need to save it initially in a special file and then call it by the solver. Here is an example in Python:
from lpsolve55 import *
from lp_maker import *
from lp_solve import *
lp = lp_maker(obj_func, constraints , rhs, sense_equality)
solvestat = lpsolve('solve', lp)
obj = lpsolve('get_objective', lp)
I think this is possible to do in C but really I don't know where to find how it is possible to do it.
One option is to use the APIs that commercial solvers like CPLEX and Gurobi provide for C/C++. Essentially, these APIs let you build the model in logical chunks (objective function, constraints, etc.). The APIs do the work of translating the logic of the model to the matrices and vectors that the solver actually needs in order to solve the model.
Another approach is to use a modeling language like AMPL or GAMS. AMPL, for example, also provides a C/C++ API.
Which one you choose probably depends on what solver you plan to use and how often you need to modify your model and/or data programmatically.
Good morning, I'm trying to perform a 2D FFT as 2 1-Dimensional FFT.
The problem setup is the following:
There's a matrix of complex numbers generated by an inverse FFT on an array of real numbers, lets call it arr[-nx..+nx][-nz..+nz].
Now, since the original array was made up of real numbers, I exploit the symmetry and reduce my array to be arr[0..nx][-nz..+nz].
My problem starts here, with arr[0..nx][-nz..nz] provided.
Now I should come back in the domain of real numbers.
The question is what kind of transformation I should use in the 2 directions?
In x I use the fftw_plan_r2r_1d( .., .., .., FFTW_HC2R, ..), called Half complex to Real transformation because in that direction I've exploited the symmetry, and that's ok I think.
But in z direction I can't figure out if I should use the same transformation or, the Complex to complex (C2C) transformation?
What is the correct once and why?
In case of needing here, at page 11, the HC2R transformation is briefly described
Thank you
"To easily retrieve a result comparable to that of fftw_plan_dft_r2c_2d(), you can chain a call to fftw_plan_dft_r2c_1d() and a call to the complex-to-complex dft fftw_plan_many_dft(). The arguments howmany and istride can easily be tuned to match the pattern of the output of fftw_plan_dft_r2c_1d(). Contrary to fftw_plan_dft_r2c_1d(), the r2r_1d(...FFTW_HR2C...) separates the real and complex component of each frequency. A second FFTW_HR2C can be applied and would be comparable to fftw_plan_dft_r2c_2d() but not exactly similar.
As quoted on the page 11 of the documentation that you judiciously linked,
'Half of these column transforms, however, are of imaginary parts, and should therefore be multiplied by I and combined with the r2hc transforms of the real columns to produce the 2d DFT amplitudes; ... Thus, ... we recommend using the ordinary r2c/c2r interface.'
Since you have an array of complex numbers, you can either use c2r transforms or unfold real/imaginary parts and try to use HC2R transforms. The former option seems the most practical.Which one might solve your issue?"
-#Francis
I have a simple task to classify people by their height and hair length to either MAN or WOMAN category using a neural network. Also teach it the pattern with some examples and then use it to classify on its own.
I have a basic understanding of neural networks but would really need some help here.
I know that each neuron divides the area to two subareas, basically that is why P = w0 + w1*x1 + w2*x2 + ... + wn*xn is being used here (weights are just moving the line if we consider geometric representation).
I do understand that each epoche should modify the weights to get closer to correct result, yet I have never program it and I am hopeless about how to start.
How should I proceed, meaning: How can I determine the threshold and how should I deal with the inputs?
It is not a homework rather than task for the ones who were interested. I am and I would like to understand it.
Looks like you are dealing with a simple Perceptron with a threshold activation function. Have a look at this question. Since you ARE using a bias neuron (w0), you would set the threshold to 0.
You then simply take the output of your network and compare it to 0, so you would e.g. output class 1 if x < 0 and class 2 if x > 0. You could model the case x=0 as "indistinct".
For learning the weights you need to apply the Delta Learning Rule which can be implemented very easily. But be careful: a perceptron with a simple threshold activation function can only be correct if your data are linearly separable. If you have more complex data you will need a Multilayer Perceptron and a nonlinear activation function like the Logistic Sigmoid Function.
Have a look at Geoffrey Hintons Coursera Course, Lecture 2 for details.
I've been working with machine learning lately (but I'm not an expert) but you should look at the Accord.NET framework. It contains all the common machine learning algorithme out of the box. So it's easy to take an existing samples and modify it instead of starting from scratch. Also, the developper of the framework is very helpful in the forum available on the same page.
With the available samples, you may also discover something better than neural network like the Kernel Support Vector Machine. If you stick to the neural network, have fun modifying all the different variables and by tryout and error you will understand how it work.
Have fun!
Since you said:
I know that each neuron divides the area to two subareas
&
weights are just moving the line if we consider geometric representation
I think you want to use perseptron or ADALINE neural networks. These neural networks can just classify linear separable patterns. since your input data is complicated, It's better to use a Multi layer Non-Linear Neural network. (my suggestion is a two layer neural network with tanh activation function) . For training these network you should use back propagation algorithm.
For answering to
how should I deal with the inputs?
I need to know more details about the inputs( Like: are they just height and hair length or there is more, what is their range and your resolution and etc.)
If you're dealing with just height and hair length I suggest that divide heights and length in some classes (for example 160cm-165cm, 165cm-170cm & etc.) and for each one of these classes set an On/Off input neuron. then put a hidden layer after all classes related to heights and another hidden layer after all classes related to hair length (tanh activation function). Number of neurons in these two hidden layer is determined based on number of training cases.
then take these two hidden layer output and send them to an aggregation layer with 1 output neuron.
I would like to get familiar with quantum computing basics.
A good way to get familiar with it would be writing very basic virtual quantum computer machines.
From what I can understand of it, the, effort of implementing a single qubit cannot simply be duplicated to implement a two qubit system. But I don't know how I would implement a single qubit either.
How do I implement a qubit?
How do I implement a set of qubits?
Example Code
If you want to start from something simple but working, you can play around with this basic quantum circuit simulator on jsfiddle (about ~2k lines, but most of that is UI stuff [drawing and clicking] and maths stuff [defining complex numbers and matrices]).
State
The state of a quantum computer is a set of complex weights, called amplitudes. There's one amplitude for each possible classical state. In the case of qubits, the classical states are just the various states a normal bit can be in.
For example, if you have three bits, then you need a complex weight for the 000, 001, 010, 011, 100, 101, 110, and 111 states.
var threeQubitState = new Complex[8];
The amplitudes must satisfy a constraint: if you add up their squared magnitudes, the result is 1. Classical states correspond to one amplitude having magnitude 1 while the others are all 0:
threeQubitState[3] = 1; // the system is 100% in the 011 state
Operations
Operations on quantum states let you redistribute the amplitude by flowing it between the classical states, but the flows you choose must preserve the squared-magnitudes-add-up-to-1 property in all cases. More technically, the operation must correspond to some unitary matrix.
var myOperation = state => new[] {
(state[1] + state[0])/sqrt(2),
(state[1] - state[0])/sqrt(2),
state[2],
state[3],
state[4],
state[5],
state[6],
state[7]
};
var myNewState = myOperation(threeQubitState);
... and those are the basics. The state is a list of complex numbers with unit 2-norm, the operations are unitary matrices, and the probability of measuring a state is just its squared amplitude.
Etc
Other things you probably need to consider:
What kinds of operations do you want to include?
A 1-qubit operation is a 2x2 matrix and a 3-qubit operation is an 8x8 matrix. How do you convert a 1-qubit operation into an 8x8 matrix when applying it to a single qubit in a 3-qubit state? (Use the Kronecker Product.)
What kinds of tricks can you use to speed up the simulation? For example, if only a few states are non-zero, or if the qubits are not entangled, there's no need to do a full matrix multiplication.
How does the user tell the simulation what to do? How can you represent what's going on for the user? There's an awful lot of numbers flowing around...
I don't actually know the answer, but an interesting place to start reading about qubits is this article. It doesn't describe in detail how entangled qubits work, but it hints at the complexity involved:
If this is how complicated things can get with only two qubits, how
complicated will it get for 3 or 4, or 100? It turns out that the
state of an N-qubit quantum computer can only be completely defined
when plotted as a point in a space with (4^N-1) dimensions. That means
we need 4^N good old fashion classical numbers to simulate it.
Note that this is the maximum space complexity, which for example is about 1 billion numbers (2^30=4^15) for 15 qubits. It says nothing about the time complexity of a simulation.
The article that #Qwertie cites is a very good introduction. If you want to implement these on your computer, you can play with the libquantum simulator, which implements sophisticated quantum operations in a C library. You can look at this example to see what using the code is like.
The information is actually stored in the interaction between different Qbits, so no implementing 1 Qbit will not translate to using multiple. I'd think another way you could play around is to use existing languages like QCL or google QCP http://qcplayground.withgoogle.com/#/home to play around
Given two recorded voices in digital format, is there an algorithm to compare the two and return a coefficient of similarity?
I recommend to take a look into the HTK toolkit for speech recognition http://htk.eng.cam.ac.uk/, especially the part on feature extraction.
Features that I would assume to be good indicators:
Mel-Cepstrum coefficients (general timbre)
LPC (for the harmonics)
Given your clarification I think what you are looking for falls under speech recognition algorithms.
Even though you are only looking for the measure of similarity and not trying to turn speech into text, still the concepts are the same and I would not be surprised if a large part of the algorithms would be quite useful.
However, you will have to define this coefficient of similarity more formally and precisely to get anywhere.
EDIT:
I believe speech recognition algorithms would be useful because they do abstraction of the sound and comparison to some known forms. Conceptually this might not be that different from taking two recordings, abstracting them and comparing them.
From wikipedia article on HMM
"In speech recognition, the hidden
Markov model would output a sequence
of n-dimensional real-valued vectors
(with n being a small integer, such as
10), outputting one of these every 10
milliseconds. The vectors would
consist of cepstral coefficients,
which are obtained by taking a Fourier
transform of a short time window of
speech and decorrelating the spectrum
using a cosine transform, then taking
the first (most significant)
coefficients."
So if you run such an algorithm on both recordings you would end up with coefficients that represent the recordings and it might be far easier to measure and establish similarities between the two.
But again now you come to the question of defining the 'similarity coefficient' and introducing dogs and horses did not really help.
(Well it does a bit, but in terms of evaluating algorithms and choosing one over another, you will have to do better).
There are many different algorithms - the general name for this task is Speaker Identification - start with this Wikipedia page and work from there: http://en.wikipedia.org/wiki/Speaker_recognition
I'm not sure this will work for soundfiles, but it gives you an idea how to proceed i hope. That is a basic way how to find a pattern (image) in another image.
You first have to calculate the fft of both the soundfiles and then do a correlation. In formular it would look like (pseudocode):
fftSoundFile1 = fft(soundFile1);
fftConjSoundFile2 = conj(fft(soundFile2));
result_corr = real(ifft(soundFile1.*soundFile2));
Where fft= fast Fourier transform, ifft = inverse, conj = conjugate complex.
The fft is performed on the sample values of the soundfiles.
The peaks in the result_corr vector will then give you the positions of high correlation.
Note that both soundfiles must in this case be of the same size-otherwise you have to place the shorter one into a file of max(soundFileLength) vector.
Regards
Edit: .* means (in matlab style) a component wise mult, you must not do a vector mult!
Next Edit: Note that you have to operate with complex numbers - but there are several Complex classes out there so I think you don't have to bother about this.