Related
EDIT: I've added the main, factorial, and trapGamma function to give the full picture but I am specifically talking about the for loop for iSum in the I function.
Basically I've run out of ideas and exhausted everywhere I know of to find an answer to this. I need to code a program that will compute a complex function which represents an M/M/1 queue.
The function includes sub functions such as calculating the integral of a gamma function and computing factorials. I've written all the code for the computations but my sum is giving me huge numbers when I would expect nothing higher than about .35
#include <math.h>
#include <stdio.h>
double I(int k, double t);
double trapGamma(double z);
unsigned long long int factorial(unsigned int n);
int main()
{
int k;
int i = 0;
double dt = 0.1;
printf("Ikx = [ \n");
for (t = 14.0 ; t <= 15.0; t += dt)
{
printf("%f " , t);
for (k = 1 ; k <= 10 ; k++)
{
I(k, t);
printf("%f " , I(k, t));
}
printf("\n");
}
printf(" ];\n");
return (0);
}
double I(int k, double t)
{
unsigned long long int x;
unsigned int n = 20;
double numerator, y, pow1, c;
double iSum;
double Ix;
int i = 0;
iSum = 0.0;
Ix = 0.0;
a = .25 * pow(t , 2);
b = pow(a, i);
x = factorial(n);
y = trapGamma(k + i + 1);
iSum = (b / (x * y));
//This is the sum loop that I'm having trouble with, I've broke the iSum equation down for my own readability while coding right above this comment
for (i = 0; i <= 100 ; i++)
{
iSum += i;
}
Ix = (pow((.5 * t), k) ) * iSum;
return Ix;
}
/*
I've checked both the factorial and trapGamma functions and they are giving me the expected results.
*/
unsigned long long int factorial(unsigned int n)
{
if(n <= 1)
return 1;
else
return (n * factorial(n - 1));
}
double trapGamma (double z)
{
int i , N = 100;
double gamma;
double a = 0.0;
double b = 15.0;
double x1, x2, y1, y2;
double areai;
double w = (b - a) / N;
gamma = 0.0;
for (i = 1; i < N; i++)
{
x1 = a + ((i - 1) * w); //the left bound point
x2 = a + (i*w); //the right bound point
y1 = pow(x1,z - 1)*exp(-x1); //the height of our left bound
y2 = pow(x2, z - 1)*exp(-x2); //the height of our right bound
areai = ((y1 + y2) / 2.0) * (x2 - x1);
gamma += areai;
}
return gamma;
}
This is building upon another project where I used a bessel function to create the M/M/1 queue over a 60 second span so I can see what this one is supposed to be. I've checked both my trapGamma and factorial functions results on there own and they are both working as expected.
How are summations supposed to be coded?
If the intent of the posted code is to calculate the modified Bessel function I, there are some pitfalls and useful semplifications to be aware of. Given
Trying to calculate the factorial, the value of the Gamma function, their product and the powers separately for each term of the sum leads to integer overflow sooner than later.
It's better to update the value of each addend of the sum instead.
Also, given that k is a whole, we have Γ(n) = (n - 1)!
The addends are increasingly smaller and, after some iterations, too small to be added to the sum, given the limited precision of type double.
// Evaluates x^k / k! trying not to overflow
double power_over_factorial(double x, int k)
{
double result = 1.0;
for ( int i = 1; i <= k; ++i )
{
result *= x / i;
}
return result;
}
#define MAX_ITERS 20
double modified_Bessel_I(int k, double x)
{
x /= 2;
const double xx = x * x;
double partial = power_over_factorial(x, k);
double old_sum, sum = partial;
int m = 1;
do
{
old_sum = sum;
partial *= xx / ((m + k) * m);
sum += partial;
}
while ( old_sum != sum && ++m < MAX_ITERS );
return sum;
}
Testable here.
I have an assignment to code a program to calculate cos(x) through the Maclaurin approximation. However I must use a function for the cos(x) and another one to calculate the exponentials that go on the denominators inside the cos(x) function. I think most of this is right, but I'm probably missing on something and I can't figure out what.
#include<stdio.h>
#include <stdlib.h>
#include <math.h>
int fat(int);
float cosx(float);
int main()
{
float x1;
/* Original code: **x1 = x1 * 3.14159 / 180;** `transforms the value to radians` */
x1 = x1 * 3.14159 / 180; /* transforms the value to radians */
printf("Insert number:\n");
scanf("%f", &x1);
printf("Cosine of %f = %f", x1, cosx(x1));
return 0;
}
int fat(int y)
{
int n, fat = 1;
for(n = 1; n <= y; n++)
{
fat = fat * n;
}
return fat;
}
float cosx(float x)
{
int i=1, a = 2, b, c = 1, e;
float cos;
while(i < 20)
{
b = c * (pow(x,a)) / e;
cos = 1 - b;
a += 2;
e = fat(a);
c *= -1;
i++;
}
return cos;
}
If I input 0 it returns -2147483648.000000, which is clearly wrong.
First error is uninitialized variable x1, and right after that you have use:
int x1; // <<< uninitiated variable;
**x1 = x1 * 3.14159 / 180;** `transforms the value to radians
this will produce random value, you should put
int x = 0; // or some other value of your choice
In my opinion you should move x1 = x1 * 3.14159/100; after scanf("%d", x1).
Than again uninitiated value e before use.
int i=1, a = 2, b, c = 1, e;
...
b = c * (pow(x,a)) / e;
...
than you have in the line b = c * pow(x,a) where you go out of range of int variable potentially. If e = 1, x = 2 and a > 31 you are out of range for b. Another problem is pow(x,a) is rising much faster than `e. thus you get bigger and bigger values thus you are getting another overflow. And here is the code that works:
#include <stdio.h>
#include <stdlib.h>
#include <math.h>
long double fact(int);
long double cosx(double);
long double my_pow (double b, int e);
int main()
{
double x1 = 45.00;
printf("Insert number:\n");
scanf("%lf", &x1);
x1 = x1 * 3.14159 / 180; // ** `transforms the value to radians`
printf("Cosine of %f = %.10LF", x1, cosx(x1));
return 0;
}
long double fact(int y)
{
int n;
double fact = 1;
for(n = 1; n <= y; n++)
{
fact *= n;
}
return fact;
}
long double cosx(double x)
{
int a = 2, c = -1;
long i = 0, lim = 500;
long double cos = 1;
long double b = 0, e = 0;
while(i < lim) {
e = fact(a);
b = c * my_pow(x,a);
cos += b/e;
// printf ("%le %le %le\n", e, b, cos);
a += 2;
c *= -1;
i++;
}
return cos;
}
long double my_pow (double b, int e) {
long double pow = 1;
for (;e > 0; --e, pow *= b)
;
return pow;
}
I'm trying to integrate the function 1/((1+x^2)x^0.5) using the trapezium rule. I need the precision to be as great as possible so I am therefore increasing the number of strips, N, until the computer cannot recognise a change between the total for consecutive N. However, the end condition is not currently working, leading to continuous integration. Does anyone have any better suggestions than my current code?
Many thanks,
Beth
#include<stdio.h>
#include<math.h>
#include<float.h>
double inter(double x, double h, double y, double N, double total)
{
total= total +0.5*(1/((1+pow(x,2))*sqrt(x)));
x=x+h;
while (x<y)
{
total=total+(1/((1+pow(x,2))*sqrt(x)));
x=x+h;
//printf("x - %.16lf \n", x);
}
total= total +0.5*(1/((1+pow(x,2))*sqrt(x)));
total=total*h;
//printf("t - %lf \n", total);
return total;
}
main()
{
double x,y,total,h,value,newvalue,f, N;
int finish;
x=0.1;
y=1000;
total=0;
N=1000;
finish=0;
value=0;
while(finish==0)
{
h=(y-x)/(N-1);
newvalue=inter(x,h,y,N,total);
printf("h-%.16lf\n", h);
printf("N-%.16lf\n", N);
printf("New value %.16lf\n", newvalue);
printf("c-%.16lf\n", value);
if(value==newvalue)
{
finish=1;
printf("finish-%d\n", finish);
}
else
{
value=newvalue;
newvalue=newvalue-3;
N=N+1000;
printf("newvalue-%lf\n", newvalue);
printf("value-%lf\n", value);
}
}
printf("%lf\n", value);
}
If you wish to create an automatic refinement of your numerical integration, one technique is to look at the relative convergence of your integration.
double previous = 0;
double current = inter( x, (y-x)/(N-1), y, N, total ); // Solve some baseline
do
{
N = N + 1000;
h = (y-x)/(N-1);
previous = current;
current = inter( x, h, y, N, total );
} while( abs( current - previous ) / current > 0.001 );
That code will stop after you observe less than 0.1% relative refinement in your estimation. Decreasing 0.001 will effectively increase your accuracy. Usually the best way to compare doubles is through a tolerance check like:
abs( a - b ) < k
where k is some factor of the order of accuracy you wish to achieve.
This integral is difficult because the f(x) -> ∞ as x -> 0. In this example, I changed the range to 1 to 1000. I also used a summation function to minimize rounding error when summing up a large number of values. The integral from wolframalpha ~= .487474, this program results in ~=.487475 . The exact integral can be found using this link:
integral 1/((1+x^2)sqrt(x))
#include<stdio.h>
#include<math.h>
#include<float.h>
/* clear array */
void clearsum(double asum[2048])
{
size_t i;
for(i = 0; i < 2048; i++)
asum[i] = 0.;
}
/* add a number into array */
void addtosum(double d, double asum[2048])
{
size_t i;
while(1){
/* i = exponent of d */
i = ((size_t)((*(unsigned long long *)&d)>>52))&0x7ff;
if(i == 0x7ff){ /* max exponent, could be overflow */
asum[i] += d;
return;
}
if(asum[i] == 0.){ /* if empty slot store d */
asum[i] = d;
return;
}
d += asum[i]; /* else add slot to d, clear slot */
asum[i] = 0.; /* and continue until empty slot */
}
}
/* return sum from array */
double returnsum(double asum[2048])
{
double sum = 0.;
size_t i;
for(i = 0; i < 2048; i++)
sum += asum[i];
return sum;
}
double fx(double x)
{
return 1./((1.+x*x)*sqrt(x));
}
double inter(double x, double y, double n)
{
double asum[2048]; /* for summation functions */
double h;
double d;
if(n < 1.){
n = 1.;
h = 0.;
} else {
h = (y-x)/(n-1.0);
}
y -= h/2.;
clearsum(asum);
d = .5*h*fx(x);
addtosum(d, asum);
for( ; x < y; x += h){
d = h*fx(x);
addtosum(d, asum);
}
d = .5*h*fx(x);
addtosum(d, asum);
d = returnsum(asum);
return d;
}
int main()
{
double x,y,n,value,newvalue;
x=1.0;
y=1000.;
value=0.;
for(n = 100000000.; 1; n += 100000000.)
{
newvalue=inter(x,y,n);
printf("new value %.16lf %.0lf\n", newvalue, n);
if(fabs(newvalue-value) < (newvalue*1E-7))
break;
value = newvalue;
}
return 0;
}
Using Simpson's rule, the results are more accurate and converge at much smaller values for n:
#include<stdio.h>
#include<math.h>
#include<float.h>
/* clear array */
void clearsum(double asum[2048])
{
size_t i;
for(i = 0; i < 2048; i++)
asum[i] = 0.;
}
/* add a number into array */
void addtosum(double d, double asum[2048])
{
size_t i;
while(1){
/* i = exponent of d */
i = ((size_t)((*(unsigned long long *)&d)>>52))&0x7ff;
if(i == 0x7ff){ /* max exponent, could be overflow */
asum[i] += d;
return;
}
if(asum[i] == 0.){ /* if empty slot store d */
asum[i] = d;
return;
}
d += asum[i]; /* else add slot to d, clear slot */
asum[i] = 0.; /* and continue until empty slot */
}
}
/* return sum from array */
double returnsum(double asum[2048])
{
double sum = 0.;
size_t i;
for(i = 0; i < 2048; i++)
sum += asum[i];
return sum;
}
double fx(double x)
{
return 1./((1.+x*x)*sqrt(x));
}
double simpson(double x, double y, double n)
{
double asum[2048]; /* for summation functions */
double h;
double a;
if(n < 1.){
n = 1.;
h = 0.;
} else {
h = (y-x)/(n-1.0);
}
y += h/2.;
clearsum(asum);
for( ; x < y; x += h){
a = h/6.*(fx(x) + 4.*fx(x + h/2.) + fx(x + h));
addtosum(a, asum);
}
a = returnsum(asum);
return a;
}
int main()
{
double x,y,n,value,newvalue;
x=1.0;
y=1000.;
value=0.;
for(n = 1000.; 1; n += 1000.)
{
newvalue=simpson(x,y,n);
printf("new value %.16lf %.0lf\n", newvalue, n);
if(fabs(newvalue-value) < (newvalue*1E-10))
break;
value = newvalue;
}
return 0;
}
I'm currently working on this : I generate a Paraview .vtm file that contains several .vtr files. Each .vtr file contains values, and coordinates, like this, assuming I'm working on a dimension of 8 :
<PointData Scalars="U">
<DataArray type="Float32" Name="U" format="ascii">
<!-- 8*8*8 values -->
</DataArray>
</PointData>
<Coordinates>
<DataArray type="Float32" Name="x" format="ascii">
<!-- 8 x values -->
</DataArray>
<DataArray type="Float32" Name="y" format="ascii">
<!-- 8 y values -->
</DataArray>
<DataArray type="Float32" Name="z" format="ascii">
<!-- 8 z values -->
</DataArray>
</Coordinates>
I use a quadridimensionnal array to store my values : float ****tab, with tab[s][x][y][z], where :
s is the current split step. It increments everytime I start working on the next .vtr file.
x, y, z the values.
Now is what causes me trouble : the coordinates where I have to place these points can be anything. It can be constant (following a step, like 0, 0.1, 0.2, and so on), or not.
I store the coordinates in three arrays : x[], y[], z[]. My goal is to cut the set of values into smaller cubes. Let's assume I split my values into 8 files (2^3 files), I have to retrieve the correct coordinates for 8 small cubes. And I can't find a way to do that.
I'm pretty sure my data structures choice is terrible, could someone give me some help with that ?
EDIT :
Here is the function generating my four-star array :
float**** fill_array_random4d(int split, int size)
{
float**** ret;
ret = malloc(sizeof(float***) * split);
for (int i = 0; i < split; i++)
{
ret[i] = malloc(sizeof (float**) * size);
for (int j = 0; j < size; j++)
{
ret[i][j] = malloc(sizeof (float*) * size);
for (int k = 0; k < size; k++)
{
ret[i][j][k] = malloc(sizeof (float) * size);
for (int l = 0; l < size; l++)
ret[i][j][k][l] = rand() % 100;
}
}
}
return ret;
}
It's a pretty basic stuff. Right now I'm using random values.
Here is how I create and fill my x, y, z arrays :
float *x, *y, *z;
x = malloc(sizeof (float) * size);
y = malloc(sizeof (float) * size);
z = malloc(sizeof (float) * size);
for (int i = 0; i < size * split; i++)
x[i] = step * i;
for (int i = 0; i < size * split; i++)
y[i] = step * i;
for (int i = 0; i < size * split; i++)
z[i] = step * i;
It's still very basic, and finally here is the function printing the coordinates in the file, following the vtk legacy format :
void print_Coordinates(FILE *file, float *x, float *y, float *z, int size, int split)
{
fprintf(file, " <Coordinates>\n");
for (int i = 0; i < 3; i++)
{
const char *text1 = " <DataArray type=\"Float32\" Name=\"";
const char *text2 = "\" format=\"ascii\">\n";
fprintf(file, "%s%c%s", text1, 'x' + i, text2);
for (int j = 0; j < size; j++)
{
if (i == 0)
fprintf(file, " %f\n", x[j]);
else if (i == 1)
fprintf(file, " %f\n", y[j]);
else
fprintf(file, " %f\n", z[j]);
}
fprintf(file, " </DataArray>\n");
}
fprintf(file, " </Coordinates>\n");
}
So, yeah, it doesn't do what I want at all.
Here is a screenshot of the result :
All the cubes are on top of each other. With the code I was using earlier, I had several cubes (one per file), but they were aligned on a diagonal (which is not good either).
As you have admitted, there are some problems with your data structure:
The first dimension s seems incongruent: Should the data structure include the original and the smaller cube? That's not easy to do, because the smaller cubes have other dimensions.
You have many separate data: The (random) data, the coordinates and the array dimensions. In order to represent the cube, you need to keep track of all of these. I recommend to create a structure to keep the relevant data together.
There isn't anything per se wrong with your approach to represent the three-dimensional array with a triple pointer, but the design leads to many fragmented allocations. A multi-dimensional array with constant dimensions is probably better represented as one "flat" memory block.
I suggest two structures:
typedef struct Cube Cube;
typedef struct Axis Axis;
struct Axis {
int n; /* number of values */
float *data; /* graduation values */
};
struct Cube {
Axis *x, *y, *z; /* Axes of the cube */
float *data; /* x-major data */
};
An "axis" stores the values along one of the axes. The cube itself doesn't worry about the axis-related code and just delegates it to its three member axes. A "cube" is your data object. (In the implementation below, the data representation is x-major, meaning the x loop is the outermost, the z loop is the innermost. You can chnage that by swapping the loops.)
If you have a populated cube object, you can the extract sub-cubes by creating a cube of a smaller dimension and copying the relevant data ranges from the axes and from the cube data. If you want to cover the whole cube, you can either extract and write the cubes as you go or store them in an array of cubes, e.g. Cube *small[8] for splitting in half for each direction. (This would be like your original s index, only that each cube may have its own dimension.)
An implementation of this behaviour with an (addmittedly simple) test main is below:
#include <stdlib.h>
#include <stdio.h>
#include <string.h>
typedef struct Cube Cube;
typedef struct Axis Axis;
struct Axis {
int n; /* number of values */
float *data; /* graduation values */
};
struct Cube {
Axis *x, *y, *z; /* Axes of the cube */
float *data; /* x-major data */
};
/*
* Create a new axis with a constant step.
*/
Axis *axis_new(int n, float start, float step)
{
Axis *axis = malloc(sizeof(*axis));
float *p;
axis->n = n;
axis->data = malloc(n * sizeof(*axis->data));
p = axis->data;
while (n--) {
*p = start;
start += step;
p++;
}
return axis;
}
/*
* Destroy and clean up axis
*/
void axis_delete(Axis *axis)
{
if (axis) {
free(axis->data);
free(axis);
}
}
/*
* Write axis in XML format to given file
*/
void axis_write(const Axis *axis, FILE *f, const char *name)
{
float *p = axis->data;
int n = axis->n;
fprintf(f, " <DataArray type=\"Float32\" "
"Name=\"%s\" format=\"ascii\">\n", name);
fprintf(f, " ");
while (n--) {
fprintf(f, " %g", *p++);
}
fprintf(f, "\n");
fprintf(f, " </DataArray>\n");
}
/*
* Create a new axis that is a sub-axis of orig.
*/
Axis *axis_slice(const Axis *orig, int start, int len)
{
Axis *axis = axis_new(len, 0, 0);
memcpy(axis->data, orig->data + start, len * sizeof(*axis->data));
return axis;
}
/*
* Create a cube of zero values for the given axes
*/
Cube *cube_new(Axis *x, Axis *y, Axis *z)
{
Cube *cube = malloc(sizeof(*cube));
int dim = x->n * y->n * z->n;
cube->x = x;
cube->y = y;
cube->z = z;
cube->data = malloc(dim * sizeof(*cube->data));
return cube;
}
/*
* Destroy and clean up cube
*/
void cube_delete(Cube *cube)
{
if (cube) {
axis_delete(cube->x);
axis_delete(cube->y);
axis_delete(cube->z);
free(cube->data);
free(cube);
}
}
float *cube_at(const Cube *cube, int x, int y, int z)
{
int pos = (x * cube->y->n + y) * cube->z->n + z;
return cube->data + pos;
}
/*
* Populate all x, y, z values according to the function func.
*/
void cube_populate(Cube *cube, float (*func)(float x, float y, float z))
{
int i, j, k;
float *p = cube->data;
for (i = 0; i < cube->x->n; i++) {
float x = cube->x->data[i];
for (j = 0; j < cube->y->n; j++) {
float y = cube->y->data[j];
for (k = 0; k < cube->z->n; k++) {
float z = cube->z->data[k];
*p++ = func(x, y, z);
}
}
}
}
/*
* Write cube to given file.
*/
void cube_write(const Cube *cube, FILE *f)
{
float *p = cube->data;
int n = cube->x->n * cube->y->n * cube->z->n;
fprintf(f, "<PointData Scalars=\"U\">\n");
fprintf(f, " <DataArray type=\"Float32\" Name=\"U\" format=\"ascii\">\n");
while (n--) {
fprintf(f, " %g", *p++);
}
fprintf(f, "\n");
fprintf(f, " </DataArray>\n");
fprintf(f, "</PointData>\n");
fprintf(f, "<Coordinates>\n");
axis_write(cube->x, f, "x");
axis_write(cube->y, f, "y");
axis_write(cube->z, f, "z");
fprintf(f, "</Coordinates>\n");
}
/*
* Create a new cube that is a sub-cube of orig.
*/
Cube *cube_slice(const Cube *orig,
int x, int dx, int y, int dy, int z, int dz)
{
Cube *cube;
float *p;
int i, j, k;
if (x + dx > orig->x->n) return NULL;
if (y + dy > orig->y->n) return NULL;
if (z + dz > orig->z->n) return NULL;
cube = cube_new(
axis_slice(orig->x, x, dx),
axis_slice(orig->y, y, dy),
axis_slice(orig->z, z, dz));
p = cube->data;
for (i = 0; i < dx; i++) {
for (j = 0; j < dy; j++) {
for (k = 0; k < dz; k++) {
*p++ = *cube_at(orig, x + i, y + j, z + k);
}
}
}
return cube;
}
/*
* Example appliaction
*/
float dist2(float x, float y, float z)
{
return x*x + y*y + z*z;
}
int main()
{
Cube *cube = cube_new(
axis_new(4, 0, 0.1),
axis_new(4, 0, 0.1),
axis_new(4, 0, 0.1));
int i, j, k;
cube_populate(cube, dist2);
for (i = 0; i < 2; i++) {
for (j = 0; j < 2; j++) {
for (k = 0; k < 2; k++) {
Cube *sub = cube_slice(cube, 2*i, 2, 2*j, 2, 2*k, 2);
cube_write(sub, stdout);
printf("--\n");
cube_delete(sub);
}
}
}
cube_delete(cube);
return 0;
}
Here is my perceptron implementation in ANSI C:
#include <stdio.h>
#include <stdlib.h>
#include <math.h>
float randomFloat()
{
srand(time(NULL));
float r = (float)rand() / (float)RAND_MAX;
return r;
}
int calculateOutput(float weights[], float x, float y)
{
float sum = x * weights[0] + y * weights[1];
return (sum >= 0) ? 1 : -1;
}
int main(int argc, char *argv[])
{
// X, Y coordinates of the training set.
float x[208], y[208];
// Training set outputs.
int outputs[208];
int i = 0; // iterator
FILE *fp;
if ((fp = fopen("test1.txt", "r")) == NULL)
{
printf("Cannot open file.\n");
}
else
{
while (fscanf(fp, "%f %f %d", &x[i], &y[i], &outputs[i]) != EOF)
{
if (outputs[i] == 0)
{
outputs[i] = -1;
}
printf("%f %f %d\n", x[i], y[i], outputs[i]);
i++;
}
}
system("PAUSE");
int patternCount = sizeof(x) / sizeof(int);
float weights[2];
weights[0] = randomFloat();
weights[1] = randomFloat();
float learningRate = 0.1;
int iteration = 0;
float globalError;
do {
globalError = 0;
int p = 0; // iterator
for (p = 0; p < patternCount; p++)
{
// Calculate output.
int output = calculateOutput(weights, x[p], y[p]);
// Calculate error.
float localError = outputs[p] - output;
if (localError != 0)
{
// Update weights.
for (i = 0; i < 2; i++)
{
float add = learningRate * localError;
if (i == 0)
{
add *= x[p];
}
else if (i == 1)
{
add *= y[p];
}
weights[i] += add;
}
}
// Convert error to absolute value.
globalError += fabs(localError);
printf("Iteration %d Error %.2f %.2f\n", iteration, globalError, localError);
iteration++;
}
system("PAUSE");
} while (globalError != 0);
system("PAUSE");
return 0;
}
The training set I'm using: Data Set
I have removed all irrelevant code. Basically what it does now it reads test1.txt file and loads values from it to three arrays: x, y, outputs.
Then there is a perceptron learning algorithm which, for some reason, is not converging to 0 (globalError should converge to 0) and therefore I get an infinite do while loop.
When I use a smaller training set (like 5 points), it works pretty well. Any ideas where could be the problem?
I wrote this algorithm very similar to this C# Perceptron algorithm:
EDIT:
Here is an example with a smaller training set:
#include <stdio.h>
#include <stdlib.h>
#include <math.h>
float randomFloat()
{
float r = (float)rand() / (float)RAND_MAX;
return r;
}
int calculateOutput(float weights[], float x, float y)
{
float sum = x * weights[0] + y * weights[1];
return (sum >= 0) ? 1 : -1;
}
int main(int argc, char *argv[])
{
srand(time(NULL));
// X coordinates of the training set.
float x[] = { -3.2, 1.1, 2.7, -1 };
// Y coordinates of the training set.
float y[] = { 1.5, 3.3, 5.12, 2.1 };
// The training set outputs.
int outputs[] = { 1, -1, -1, 1 };
int i = 0; // iterator
FILE *fp;
system("PAUSE");
int patternCount = sizeof(x) / sizeof(int);
float weights[2];
weights[0] = randomFloat();
weights[1] = randomFloat();
float learningRate = 0.1;
int iteration = 0;
float globalError;
do {
globalError = 0;
int p = 0; // iterator
for (p = 0; p < patternCount; p++)
{
// Calculate output.
int output = calculateOutput(weights, x[p], y[p]);
// Calculate error.
float localError = outputs[p] - output;
if (localError != 0)
{
// Update weights.
for (i = 0; i < 2; i++)
{
float add = learningRate * localError;
if (i == 0)
{
add *= x[p];
}
else if (i == 1)
{
add *= y[p];
}
weights[i] += add;
}
}
// Convert error to absolute value.
globalError += fabs(localError);
printf("Iteration %d Error %.2f\n", iteration, globalError);
}
iteration++;
} while (globalError != 0);
// Display network generalisation.
printf("X Y Output\n");
float j, k;
for (j = -1; j <= 1; j += .5)
{
for (j = -1; j <= 1; j += .5)
{
// Calculate output.
int output = calculateOutput(weights, j, k);
printf("%.2f %.2f %s\n", j, k, (output == 1) ? "Blue" : "Red");
}
}
// Display modified weights.
printf("Modified weights: %.2f %.2f\n", weights[0], weights[1]);
system("PAUSE");
return 0;
}
In your current code, the perceptron successfully learns the direction of the decision boundary BUT is unable to translate it.
y y
^ ^
| - + \\ + | - \\ + +
| - +\\ + + | - \\ + + +
| - - \\ + | - - \\ +
| - - + \\ + | - - \\ + +
---------------------> x --------------------> x
stuck like this need to get like this
(as someone pointed out, here is a more accurate version)
The problem lies in the fact that your perceptron has no bias term, i.e. a third weight component connected to an input of value 1.
w0 -----
x ---->| |
| f |----> output (+1/-1)
y ---->| |
w1 -----
^ w2
1(bias) ---|
The following is how I corrected the problem:
#include <stdio.h>
#include <stdlib.h>
#include <math.h>
#include <time.h>
#define LEARNING_RATE 0.1
#define MAX_ITERATION 100
float randomFloat()
{
return (float)rand() / (float)RAND_MAX;
}
int calculateOutput(float weights[], float x, float y)
{
float sum = x * weights[0] + y * weights[1] + weights[2];
return (sum >= 0) ? 1 : -1;
}
int main(int argc, char *argv[])
{
srand(time(NULL));
float x[208], y[208], weights[3], localError, globalError;
int outputs[208], patternCount, i, p, iteration, output;
FILE *fp;
if ((fp = fopen("test1.txt", "r")) == NULL) {
printf("Cannot open file.\n");
exit(1);
}
i = 0;
while (fscanf(fp, "%f %f %d", &x[i], &y[i], &outputs[i]) != EOF) {
if (outputs[i] == 0) {
outputs[i] = -1;
}
i++;
}
patternCount = i;
weights[0] = randomFloat();
weights[1] = randomFloat();
weights[2] = randomFloat();
iteration = 0;
do {
iteration++;
globalError = 0;
for (p = 0; p < patternCount; p++) {
output = calculateOutput(weights, x[p], y[p]);
localError = outputs[p] - output;
weights[0] += LEARNING_RATE * localError * x[p];
weights[1] += LEARNING_RATE * localError * y[p];
weights[2] += LEARNING_RATE * localError;
globalError += (localError*localError);
}
/* Root Mean Squared Error */
printf("Iteration %d : RMSE = %.4f\n",
iteration, sqrt(globalError/patternCount));
} while (globalError > 0 && iteration <= MAX_ITERATION);
printf("\nDecision boundary (line) equation: %.2f*x + %.2f*y + %.2f = 0\n",
weights[0], weights[1], weights[2]);
return 0;
}
... with the following output:
Iteration 1 : RMSE = 0.7206
Iteration 2 : RMSE = 0.5189
Iteration 3 : RMSE = 0.4804
Iteration 4 : RMSE = 0.4804
Iteration 5 : RMSE = 0.3101
Iteration 6 : RMSE = 0.4160
Iteration 7 : RMSE = 0.4599
Iteration 8 : RMSE = 0.3922
Iteration 9 : RMSE = 0.0000
Decision boundary (line) equation: -2.37*x + -2.51*y + -7.55 = 0
And here's a short animation of the code above using MATLAB, showing the decision boundary at each iteration:
It might help if you put the seeding of the random generator at the start of your main instead of reseeding on every call to randomFloat, i.e.
float randomFloat()
{
float r = (float)rand() / (float)RAND_MAX;
return r;
}
// ...
int main(int argc, char *argv[])
{
srand(time(NULL));
// X, Y coordinates of the training set.
float x[208], y[208];
Some small errors I spotted in your source code:
int patternCount = sizeof(x) / sizeof(int);
Better change this to
int patternCount = i;
so you doesn't have to rely on your x array to have the right size.
You increase iterations inside the p loop, whereas the original C# code does this outside the p loop. Better move the printf and the iteration++ outside the p loop before the PAUSE statement - also I'd remove the PAUSE statement or change it to
if ((iteration % 25) == 0) system("PAUSE");
Even doing all those changes, your program still doesn't terminate using your data set, but the output is more consistent, giving an error oscillating somewhere between 56 and 60.
The last thing you could try is to test the original C# program on this dataset, if it also doesn't terminate, there's something wrong with the algorithm (because your dataset looks correct, see my visualization comment).
globalError will not become zero, it will converge to zero as you said, i.e. it will become very small.
Change your loop like such:
int maxIterations = 1000000; //stop after one million iterations regardless
float maxError = 0.001; //one in thousand points in wrong class
do {
//loop stuff here
//convert to fractional error
globalError = globalError/((float)patternCount);
} while ((globalError > maxError) && (i<maxIterations));
Give maxIterations and maxError values applicable to your problem.