Solving a coupled differential equations system using time splitting - c

/******************************************************************************
Online C Compiler.
Code, Compile, Run and Debug C program online.
Write your code in this editor and press "Run" button to compile and execute it.
*******************************************************************************/
#include <stdlib.h>
#include <stdio.h>
#include <math.h>
#include <string.h>
#define PI 3.141592
void read_input(double *D, double *L, int *nx, double *t_F);
double main(void) {
/******************************/
/* Declarations of parameters */
/******************************/
/* Number of grid points */
int nx;
/* Length of domain */
double L;
/* Equation coefficients */
double D;
/* Length of time to run simulation. */
double t_F;
/* Read in from file; */
read_input(&D, &L, &nx, &t_F);
/* Grid spacing */
double dx = L/nx;
double invdx2 = 1.0/(dx*dx);
/* Time step */
double dt = 0.25/invdx2; // changed to 0.25/dx^2 to satisfy the stability condition
/************************************************/
/* Solution Storage at Current / Next time step */
/************************************************/
double *uc, *un, *vc, *vn;
/* Time splitting solutions */
double *uts1, *uts2, *vts1, *vts2;
/* Derivative used in finite difference */
double deriv;
/* Allocate memory according to size of nx */
uc = malloc(nx * sizeof(double));
un = malloc(nx * sizeof(double));
vc = malloc(nx * sizeof(double));
vn = malloc(nx * sizeof(double));
uts1 = malloc(nx * sizeof(double));
uts2 = malloc(nx * sizeof(double));
vts1 = malloc(nx * sizeof(double));
vts2 = malloc(nx * sizeof(double));
/* Check the allocation pointers */
if (uc==NULL||un==NULL||vc==NULL||vn==NULL||uts1==NULL||
uts2==NULL||vts1==NULL||vts2==NULL) {
printf("Memory allocation failed\n");
return 1;
}
int k;
double x;
/* Current time */
double ctime;
/* Initialise arrays */
for(k = 0; k < nx; k++) {
x = k*dx;
uc[k] = 1.0 + sin(2.0*PI*x/L);
vc[k] = 0.0;
/* Set other arrays to 0 */
uts1[k] = 0; uts2[k] = 0;
vts1[k] = 0; vts2[k] = 0;
}
/* Loop over timesteps */
while (ctime < t_F){
/* Rotation factors for time-splitting scheme. */
double cfac = cos(dt); //changed from 2*dt to dt
double sfac = sin(dt);
/* First substep for diffusion equation, A_1 */
for (k = 0; k < nx; k++) {
x = k*dx;
/* Diffusion at half time step. */
deriv = (uc[k-1] + uc[k+1] - 2*uc[k])*invdx2 ;
uts1[k] = uc[k] + (D * deriv + vc[k])* 0.5*dt; //
deriv = (vc[k-1] + vc[k+1] - 2*vc[k])*invdx2;
vts1[k] = vc[k] + (D * deriv - uc[k]) * 0.5*dt;
}
/* Second substep for decay/growth terms, A_2 */
for (k = 0; k < nx; k++) {
x = k*dx;
/* Apply rotation matrix to u and v, */
uts2[k] = cfac*uts1[k] + sfac*vts1[k];
vts2[k] = -sfac*uts1[k] + cfac*vts1[k];
}
/* Third substep for diffusion terms, A_1 */
for (k = 0; k < nx; k++) {
x = k*dx;
deriv = (uts2[k-1] + uts2[k+1] - 2*uts2[k])*invdx2;
un[k] = uts2[k] + (D * deriv + vts2[k]) * 0.5*dt;
deriv = (vts2[k-1] + vts2[k+1] - 2*vts2[k])*invdx2;
vn[k] = vts2[k] + (D * deriv - uts2[k]) * 0.5*dt;
}
/* Copy next values at timestep to u, v arrays. */
memcpy(uc,un, sizeof(double) * nx);
memcpy(vc,vn, sizeof(double) * nx);
/* Increment time. */
ctime += dt;
for (k = 0; k < nx; k++ ) {
x = k*dx;
printf("%g %g %g %g\n",ctime,x,uc[k],vc[k]);
}
}
/* Free allocated memory */
free(uc); free(un);
free(vc); free(vn);
free(uts1); free(uts2);
free(vts1); free(vts2);
return 0;
}
// The lines below don't contain any bugs! Don't modify them
void read_input(double *D, double *L, int *nx, double *t_F) {
FILE *infile;
if(!(infile=fopen("input.txt","r"))) {
printf("Error opening file\n");
exit(1);
}
if(4!=fscanf(infile,"%lf %lf %d %lf",D,L,nx,t_F)) {
printf("Error reading parameters from file\n");
exit(1);
}
fclose(infile);
}
So this is the code. It is meant to solve the following differential equations:
du/dt - Dd^2u/dx^2 - v = 0
dv/dt - Dd^2v/dx^2 + u = 0
It splits the equations into two parts. The second x derivative part(A1) and the decay part which contains u and v(A2) . It uses two half steps(0.5dt) for A1 and 1 full step of dt for A2. I know how to do time splitting but i dont know whether i have done it correctly here.
This is for an assignment and i have fixed all the errors and i am just trying to make the code work as intended. I have never had to solve something similar to this so i am definitely very stuck right now. The solution converges but i think its wrong. Any ideas why? Am not looking for someone to outright tell me what am doing wrong, just guide me in the right direction if you know what i mean.
PS: When i compile the code with gcc i get a warning about double main(void). Why might that be?

Related

Writing a wave generator with SDL

I've coded a simple sequencer in C with SDL 1.2 and SDL_mixer(to play .wav file). It works well and I want to add some audio synthesis to this program. I've look up the and I found this sinewave code using SDL2(https://github.com/lundstroem/synth-samples-sdl2/blob/master/src/synth_samples_sdl2_2.c)
Here's how the sinewave is coded in the program:
static void build_sine_table(int16_t *data, int wave_length)
{
/*
Build sine table to use as oscillator:
Generate a 16bit signed integer sinewave table with 1024 samples.
This table will be used to produce the notes.
Different notes will be created by stepping through
the table at different intervals (phase).
*/
double phase_increment = (2.0f * pi) / (double)wave_length;
double current_phase = 0;
for(int i = 0; i < wave_length; i++) {
int sample = (int)(sin(current_phase) * INT16_MAX);
data[i] = (int16_t)sample;
current_phase += phase_increment;
}
}
static double get_pitch(double note) {
/*
Calculate pitch from note value.
offset note by 57 halfnotes to get correct pitch from the range we have chosen for the notes.
*/
double p = pow(chromatic_ratio, note - 57);
p *= 440;
return p;
}
static void audio_callback(void *unused, Uint8 *byte_stream, int byte_stream_length) {
/*
This function is called whenever the audio buffer needs to be filled to allow
for a continuous stream of audio.
Write samples to byteStream according to byteStreamLength.
The audio buffer is interleaved, meaning that both left and right channels exist in the same
buffer.
*/
// zero the buffer
memset(byte_stream, 0, byte_stream_length);
if(quit) {
return;
}
// cast buffer as 16bit signed int.
Sint16 *s_byte_stream = (Sint16*)byte_stream;
// buffer is interleaved, so get the length of 1 channel.
int remain = byte_stream_length / 2;
// split the rendering up in chunks to make it buffersize agnostic.
long chunk_size = 64;
int iterations = remain/chunk_size;
for(long i = 0; i < iterations; i++) {
long begin = i*chunk_size;
long end = (i*chunk_size) + chunk_size;
write_samples(s_byte_stream, begin, end, chunk_size);
}
}
static void write_samples(int16_t *s_byteStream, long begin, long end, long length) {
if(note > 0) {
double d_sample_rate = sample_rate;
double d_table_length = table_length;
double d_note = note;
/*
get correct phase increment for note depending on sample rate and table length.
*/
double phase_increment = (get_pitch(d_note) / d_sample_rate) * d_table_length;
/*
loop through the buffer and write samples.
*/
for (int i = 0; i < length; i+=2) {
phase_double += phase_increment;
phase_int = (int)phase_double;
if(phase_double >= table_length) {
double diff = phase_double - table_length;
phase_double = diff;
phase_int = (int)diff;
}
if(phase_int < table_length && phase_int > -1) {
if(s_byteStream != NULL) {
int16_t sample = sine_wave_table[phase_int];
sample *= 0.6; // scale volume.
s_byteStream[i+begin] = sample; // left channel
s_byteStream[i+begin+1] = sample; // right channel
}
}
}
}
}
I don't understand how I could change the sinewave formula to genrate other waveform like square/triangle/saw ect...
EDIT:
Because I forgot to explain it, here's what I tried.
I followed the example I've seen on this video series(https://www.youtube.com/watch?v=tgamhuQnOkM). The source code of the method provided by the video is on github, and the wave generation code is looking like this:
double w(double dHertz)
{
return dHertz * 2.0 * PI;
}
// General purpose oscillator
double osc(double dHertz, double dTime, int nType = OSC_SINE)
{
switch (nType)
{
case OSC_SINE: // Sine wave bewteen -1 and +1
return sin(w(dHertz) * dTime);
case OSC_SQUARE: // Square wave between -1 and +1
return sin(w(dHertz) * dTime) > 0 ? 1.0 : -1.0;
case OSC_TRIANGLE: // Triangle wave between -1 and +1
return asin(sin(w(dHertz) * dTime)) * (2.0 / PI);
}
Because the C++ code here uses windows soun api I could not copy/paste this method to make it work on the piece of code I've found using SDL2.
So I tried to this in order to obtain a square wave:
static void build_sine_table(int16_t *data, int wave_length)
{
double phase_increment = ((2.0f * pi) / (double)wave_length) > 0 ? 1.0 : -1.0;
double current_phase = 0;
for(int i = 0; i < wave_length; i++) {
int sample = (int)(sin(current_phase) * INT16_MAX);
data[i] = (int16_t)sample;
current_phase += phase_increment;
}
}
This didn't gave me a square wave but more a saw wave.
Here's what I tried to get a triangle wave:
static void build_sine_table(int16_t *data, int wave_length)
{
double phase_increment = (2.0f * pi) / (double)wave_length;
double current_phase = 0;
for(int i = 0; i < wave_length; i++) {
int sample = (int)(asin(sin(current_phase) * INT16_MAX)) * (2 / pi);
data[i] = (int16_t)sample;
current_phase += phase_increment;
}
}
This also gave me another type of waveform, not triangle.
You’d replace the sin function call with call to one of the following:
// this is a helper function only
double normalize(double phase)
{
double cycles = phase/(2.0*M_PI);
phase -= trunc(cycles) * 2.0 * M_PI;
if (phase < 0) phase += 2.0*M_PI;
return phase;
}
double square(double phase)
{ return (normalize(phase) < M_PI) ? 1.0 : -1.0; }
double sawtooth(double phase)
{ return -1.0 + normalize(phase) / M_PI; }
double triangle(double phase)
{
phase = normalize(phase);
if (phase >= M_PI)
phase = 2*M_PI - phase;
return -1.0 + 2.0 * phase / M_PI;
}
You’d be building tables just like you did for the sine, except they’d be the square, sawtooth and triangle tables, respectively.

Euler's method for neuroscience in C [closed]

Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 4 years ago.
Improve this question
I need to estimate for my internship the firing rate of neurons that follows an ODE. I code at first in python and it goes pretty well but for better performance, my supervisor told to write the same code in C. However, I never coded in C so I am a very beginner and the file in which i want to have the values for the firing rate is full of zero... Can someone help me ?
Thank you very much
So here is my python code :
import numpy as np
import matplotlib.pyplot as plt
from math import cos, sin, sqrt, pi, exp as cos, sin, sqrt, pi, exp
#parameters
eps = 0.05
f = 0.215
mu = 1.1
D = 0.001
DeltaT = 0.01
timewindow = 40
num_points = int(timewindow/DeltaT)
T = np.linspace(0, timewindow, num_points)
#signal
s = [sin(2*3.14*f*t) for t in T]
N=30000
compteur=np.zeros(num_points)
v = np.zeros((num_points,N))
samples = np.random.normal(0, 1, (num_points,N))
for i in range(1,num_points):
for j in range(N):
v[i,j] = v[i-1,j] + DeltaT *(-v[i-1,j]+ mu + eps*s[i-1]) + \
sqrt(2*D*DeltaT)*samples[i,j]
if v[i,j]>1:
v[i,j]=0
compteur[i]+=1/DeltaT/N
plt.plot(T,compteur)
plt.show()
and here is my "translation" in C :
#include <math.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#define PI 3.1415926536
float s(float x, float);
double AWGN_generator();
FILE* fopen(const char* nomDuFichier, const char* modeOuverture);
int fclose(FILE* pointeurSurFichier);
int main(int argc, char *argv[])
{
//parameters
double eps = 0.05;
float f = 0.215 ;
double mu = 1.1 ;
double D = 0.001 ;
int time_window = 90;
int num_points = 1000;
long num_neurons = 1000;
double deltaT = time_window/num_points;
int i ;
//time
double Time[num_points] ;
Time[0]= 0.0;
for (i = 1 ; i < num_points ; i++ )
{ Time[i] = Time[i-1] + deltaT;
}
//opening file for saving data
FILE* fichier = NULL;
fichier = fopen("challala.txt", "w");
if (fichier != NULL)
{
double v[num_points][num_neurons] ;
memset(v, 0, num_points*num_neurons*sizeof(long) );
long compteur[num_points];
memset( compteur, 0, num_points*sizeof(long) );
int pos_1 ;
int pos_2 ;
//Euler's method
for (pos_1 = 1 ; pos_1 < num_points ; pos_1 ++)
{
for (pos_2 = 0 ; pos_2<num_neurons ; pos_2 ++)
{
float t = Time[pos_1-1] ;
v[pos_1][pos_2] = v[pos_1-1][pos_2] + deltaT *(-v[pos_1-1]
[pos_2]+ mu + eps*s(t, f))+ sqrt(2*D*deltaT)*AWGN_generator();
if (v[pos_1][pos_2]>1)
{
v[pos_1][pos_2]=0 ;
compteur[pos_1]+=1/deltaT/num_neurons ;
}
}
fprintf(fichier, "%ld",compteur[pos_1]);
}
fclose(fichier);
printf("ca a marche test.txt");
}
else
{
// On affiche un message d'erreur si on veut
printf("Impossible d'ouvrir le fichier test.txt");
}
return 0;
}
float s(float x, float f)
{
return sin(2*M_PI*f*x);
}
double AWGN_generator()
{/* Generates additive white Gaussian Noise samples with zero mean and a
standard deviation of 1. */
double temp1;
double temp2;
double result;
int p;
p = 1;
while( p > 0 )
{
temp2 = ( rand() / ( (double)RAND_MAX ) ); /* rand() function generates an
integer between 0 and
RAND_MAX,
which is defined in
stdlib.h.
*/
if ( temp2 == 0 )
{// temp2 is >= (RAND_MAX / 2)
p = 1;
}// end if
else
{// temp2 is < (RAND_MAX / 2)
p = -1;
}// end else
}// end while()
temp1 = cos( ( 2.0 * (double)PI ) * rand() / ( (double)RAND_MAX ) );
result = sqrt( -2.0 * log( temp2 ) ) * temp1;
return result; // return the generated random sample to the caller
}
Code repaired as able with old code commented out. See comments for change details.
Need to see challala.txt for a definitive test.
#include <math.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
// Why use a coarse approximation?
//#define PI 3.1415926536
#define PI 3.1415926535897932384626433832795
// Let us stick to double only.
//float s(float x, float);
double s(double x, double);
// Add void, else declaration does not check the parameters.
//double AWGN_generator();
double AWGN_generator(void);
// These should have already been declared in <stdio.h>
// FILE* fopen(const char* nomDuFichier, const char* modeOuverture);
// int fclose(FILE* pointeurSurFichier);
int main(int argc, char *argv[]) {
//parameters
double eps = 0.05;
// float f = 0.215;
double f = 0.215;
double mu = 1.1;
double D = 0.001;
int time_window = 90;
// Unclear why `int/long` used here. size_t would be idiomatic for array sizing.
int num_points = 1000;
long num_neurons = 1000;
// Avoid integer division when a FP quotinet is desired
// double deltaT = time_window / num_points;
double deltaT = 1.0*time_window / num_points;
int i;
//time
double Time[num_points];
Time[0] = 0.0;
for (i = 1; i < num_points; i++)
{
Time[i] = Time[i - 1] + deltaT;
}
//opening file for saving data
FILE* fichier = NULL;
fichier = fopen("challala.txt", "w");
if (fichier != NULL) {
double v[num_points][num_neurons];
// zero fill use wrong type in sizeof.
// Avoid type in sizeof, better to use sizeof object
// memset(v, 0, num_points * num_neurons * sizeof(long));
memset(v, 0, sizeof v);
// Let us use FP here.
//long compteur[num_points];
double compteur[num_points];
memset(compteur, 0, sizeof compteur);
int pos_1;
int pos_2;
//Euler's method
for (pos_1 = 1; pos_1 < num_points; pos_1++)
{
for (pos_2 = 0; pos_2 < num_neurons; pos_2++) {
// float t = Time[pos_1 - 1];
double t = Time[pos_1 - 1];
v[pos_1][pos_2] = v[pos_1 - 1][pos_2]
+ deltaT * (-v[pos_1 - 1][pos_2] + mu + eps * s(t, f))
+ sqrt(2 * D * deltaT) * AWGN_generator();
if (v[pos_1][pos_2] > 1) {
v[pos_1][pos_2] = 0;
compteur[pos_1] += 1 / deltaT / num_neurons;
}
}
// Change of type
// fprintf(fichier, "%ld", compteur[pos_1]);
fprintf(fichier, " %g", compteur[pos_1]);
}
fclose(fichier);
printf("ca a marche test.txt");
} else {
// Was not the file another name?
// printf("Impossible d'ouvrir le fichier test.txt");
printf("Impossible d'ouvrir le fichier \"%s\"\n", challala.txt);
}
return 0;
}
//float s(float x, float f) {
double s(double x, double f) {
// M_PI is not defined in the standard C library, although common in extensions.
//return sin(2 * M_PI * f * x);
return sin(2 * PI * f * x);
}
double AWGN_generator() {
double temp1;
double temp2;
double result;
int p;
p = 1;
while (p > 0) {
temp2 = (rand() / ((double) RAND_MAX));
if (temp2 == 0) { // temp2 is >= (RAND_MAX / 2)
p = 1;
} // end if
else { // temp2 is < (RAND_MAX / 2)
p = -1;
} // end else
} // end while()
temp1 = cos((2.0 * (double) PI) * rand() / ((double) RAND_MAX));
result = sqrt(-2.0 * log(temp2)) * temp1;
return result; // return the generated random sample to the caller
}
Minor and advanced numeric issue:
Realize that quotient 1.0*time_window / num_points maybe be a little different than mathematical expected due to finite precision of double. This is, at worst, expected to be a very small amount, maybe about 0.5 parts in 253.
Yet the repetitive additions accumulate the error.
double deltaT = 1.0*time_window / num_points;
int i;
double Time[num_points];
Time[0] = 0.0;
for (i = 1; i < num_points; i++) {
Time[i] = Time[i - 1] + deltaT;
}
To avoid that accumulated error, code can re-calculate Time[i] anew on each iteration.
double deltaT = 1.0*time_window / num_points;
double Time[num_points];
for (int i = 0; i < num_points; i++) {
Time[i] = deltaT*i;
}
Of course, such small errors are often ignorable, but mayne not when num_points is large enough. This happens when your good code is applied to ever larger tasks.
regarding the following 3 statements
int time_window = 90;
int num_points = 1000;
double deltaT = time_window/num_points;
Since time_window and num_points are integers, the division is performed as an integer divide.
In an integer divide, all fractions are truncated.
the expression: time_window/num_points is actually:
90 / 1000
the resulting fraction has everything right of the decimal point truncated, so the result is 0
so: Time[0] + 0 results in 0.0.
The same (calculated) value: 0.0 is then propagated thorough the whole array
suggest changing:
int time_window = 90;
int num_points = 1000;
to
double time_window = 90.0;
double num_points = 1000.0;
regarding:
memset(v, 0, num_points*num_neurons*sizeof(long) );
this statement may (or may not) perform the desired functionality. It depends on if the size of double is the same as the size of long
Suggest using:
memset( v, 0, sizeof( v ) );

How to use GSL to fit an arbitrary function (i.e. 1/x + 1/x^2) to some data?

I have some data and I need to fit a second order "polynomial" in 1/x to it using C and GSL, but I don't really understand how to do it.
The documentation for GSL is, unfortunately, not very helpful, I have read it for a few hours now, but I don't seem to be getting closer to the solution.
Google doesn't turn up anything useful either, and I really don't know what to do anymore.
Could you maybe give me some hints on how to accomplish this, or where even to look?
Thanks
Edit 1: The main problem basically is that
Sum n : a_n*x^(-1)
is not a polynomial, so basic fitting or solving algorithms won't work correctly. That's what I tried, using the code for quadratic fitting from this link, also substituting x->1/x, but it didn't work.
May be it's a bit too late for you to read this. However, I post my answer anyway for other people looking for enlightenment.
I suppose, that this basic example can help you. First of all, you have to read about this method of non-linear fitting since you have to adapt the code for any of your own problem.
Second, it's a bit not really clear for me from your post what function you use.
For the sake of clarity let's consider
a1/x + a2/x**2
where a1 and a2 - your parameters.
Using that slightly modified code from the link above ( I replaced 1/x with 1/(x + 0.1) to avoid singularities but it doesn't really change the picture):
#include <stdlib.h>
#include <stdio.h>
#include <gsl/gsl_rng.h>
#include <gsl/gsl_randist.h>
#include <gsl/gsl_matrix.h>
#include <gsl/gsl_vector.h>
#include <gsl/gsl_blas.h>
#include <gsl/gsl_multifit_nlinear.h>
/* number of data points to fit */
#define N 40
#define FIT(i) gsl_vector_get(w->x, i)
#define ERR(i) sqrt(gsl_matrix_get(covar,i,i))
struct data
{
size_t n;
double * y;
};
int expb_f (const gsl_vector * x, void *data, gsl_vector * f)
{
size_t n = ((struct data *)data)->n;
double *y = ((struct data *)data)->y;
double A_1 = gsl_vector_get (x, 0);
double A_2 = gsl_vector_get (x, 1);
size_t i;
for (i = 0; i < n; i++)
{
/* Model Yi = A_1 / x + A_2 / x**2 */
double t = i;
double Yi = A_1 / (t + 0.1) +A_2 / (t*t + 0.2*t + 0.01) ;
gsl_vector_set (f, i, Yi - y[i]);
}
return GSL_SUCCESS;
}
int expb_df (const gsl_vector * x, void *data, gsl_matrix * J)
{
size_t n = ((struct data *)data)->n;
double A_1 = gsl_vector_get (x, 0);
double A_2 = gsl_vector_get (x, 1);
size_t i;
for (i = 0; i < n; i++)
{
/* Jacobian matrix J(i,j) = dfi / dxj, */
/* where fi = (Yi - yi)/sigma[i], */
/* Yi = A_1 / (t + 0.1) +A_2 / (t*t + 0.2*t + 0.01) */
/* and the xj are the parameters (A_1,A_2) */
double t = i;
double e = 1 / (t + 0.1);
double e1 = 1 / (t*t + 0.2*t + 0.01);
gsl_matrix_set (J, i, 0, e);
gsl_matrix_set (J, i, 1, e1);
}
return GSL_SUCCESS;
}
void callback(const size_t iter, void *params, const gsl_multifit_nlinear_workspace *w)
{
gsl_vector *f = gsl_multifit_nlinear_residual(w);
gsl_vector *x = gsl_multifit_nlinear_position(w);
double rcond;
/* compute reciprocal condition number of J(x) */
gsl_multifit_nlinear_rcond(&rcond, w);
fprintf(stderr, "iter %2zu: A_1 = % e A_2 = % e cond(J) = % e, |f(x)| = % e \n", iter, gsl_vector_get(x, 0), gsl_vector_get(x, 1), 1.0 / rcond, gsl_blas_dnrm2(f));
}
int main (void)
{
const gsl_multifit_nlinear_type *T = gsl_multifit_nlinear_trust;
gsl_multifit_nlinear_workspace *w;
gsl_multifit_nlinear_fdf fdf;
gsl_multifit_nlinear_parameters fdf_params = gsl_multifit_nlinear_default_parameters();
const size_t n = N;
const size_t p = 2;
gsl_vector *f;
gsl_matrix *J;
gsl_matrix *covar = gsl_matrix_alloc (p, p);
double y[N], weights[N];
struct data d = { n, y };
double x_init[2] = { 1.0, 1.0 }; /* starting values */
gsl_vector_view x = gsl_vector_view_array (x_init, p);
gsl_vector_view wts = gsl_vector_view_array(weights, n);
gsl_rng * r;
double chisq, chisq0;
int status, info;
size_t i;
const double xtol = 1e-8;
const double gtol = 1e-8;
const double ftol = 0.0;
gsl_rng_env_setup();
r = gsl_rng_alloc(gsl_rng_default);
/* define the function to be minimized */
fdf.f = expb_f;
fdf.df = expb_df; /* set to NULL for finite-difference Jacobian */
fdf.fvv = NULL; /* not using geodesic acceleration */
fdf.n = n;
fdf.p = p;
fdf.params = &d;
/* this is the data to be fitted */
for (i = 0; i < n; i++)
{
double t = i;
double yi = (0.1 + 3.2/(t + 0.1))/(t + 0.1);
double si = 0.1 * yi;
double dy = gsl_ran_gaussian(r, si);
weights[i] = 1.0 / (si * si);
y[i] = yi + dy;
printf ("% e % e \n",t + 0.1, y[i]);
};
/* allocate workspace with default parameters */
w = gsl_multifit_nlinear_alloc (T, &fdf_params, n, p);
/* initialize solver with starting point and weights */
gsl_multifit_nlinear_winit (&x.vector, &wts.vector, &fdf, w);
/* compute initial cost function */
f = gsl_multifit_nlinear_residual(w);
gsl_blas_ddot(f, f, &chisq0);
/* solve the system with a maximum of 20 iterations */
status = gsl_multifit_nlinear_driver(20, xtol, gtol, ftol, callback, NULL, &info, w);
/* compute covariance of best fit parameters */
J = gsl_multifit_nlinear_jac(w);
gsl_multifit_nlinear_covar (J, 0.0, covar);
/* compute final cost */
gsl_blas_ddot(f, f, &chisq);
fprintf(stderr, "summary from method '%s/%s'\n", gsl_multifit_nlinear_name(w), gsl_multifit_nlinear_trs_name(w));
fprintf(stderr, "number of iterations: %zu \n", gsl_multifit_nlinear_niter(w));
fprintf(stderr, "function evaluations: %zu \n", fdf.nevalf);
fprintf(stderr, "Jacobian evaluations: %zu \n", fdf.nevaldf);
fprintf(stderr, "reason for stopping: %s \n", (info == 1) ? "small step size" : "small gradient");
fprintf(stderr, "initial |f(x)| = % e \n", sqrt(chisq0));
fprintf(stderr, "final |f(x)| = % e \n", sqrt(chisq));
{
double dof = n - p;
double c = GSL_MAX_DBL(1, sqrt(chisq / dof));
fprintf(stderr, "chisq/dof = % e \n", chisq / dof);
fprintf (stderr, "A_1 = % f +/- % f \n", FIT(0), c*ERR(0));
fprintf (stderr, "A_2 = % f +/- % f \n", FIT(1), c*ERR(1));
}
fprintf (stderr, "status = %s \n", gsl_strerror (status));
gsl_multifit_nlinear_free (w);
gsl_matrix_free (covar);
gsl_rng_free (r);
return 0;
}
Results of simulations
Unfortunately, Gnuplot doesn't want to fit this data for some reason. Usually it gives the same function up to certain decimal numbers and helps to verify your code.

arctan(x) function gives wrong answer

I'm using the Maclaurin series for arctan(x) and I am not getting the correct answer. I'm doing the calculation in radians. Here's the function so far:
fp32 t32rArcTangent(fp32 number)
{
fp32 a, b, c, d; /* Temp Variables */
fp32 t; /* Number Temp */
uint32 i; /* Loop Counter */
/* Time Savers */
if (b32fpcomp(number, MM_FP8INFINITY)) return((fp32)MM_PI / 2);
if (b32fpcomp(number, -MM_FP8INFINITY)) return(-(fp32)MM_PI / 2);
/* Setup */
a = 0;
b = 0;
c = 1;
d = number;
t = number * number;
/* Calculation Loop */
for (i = 0; i < MMPRVT_FP32_TRIG_LIMIT; i++)
{
b += d;
if (b32fpcomp(a, b)) break;
a = b;
c += 2;
d *= -1 * t / c;
}
#ifdef DEBUG
printf("Loops: %lu\n", i);
#endif
/* Result */
return(a);
fp32 = typedef'd float
uint32 = typedef'd unsigned long int
MM_FP8INFINITY is the largest number that the fp32 datatype can contain.
MM_PI is just PI out to about 50 digits.
MMPRVT_FP32_TRIG_LIMIT is the maximum number of loops that can be used to calculate the result. This is to prevent the series expansion from going into an infinite loop if for whatever reason the series fails to converge.
These are the results that I am getting:
Testing arctangent(x) function.
Loops: 0
arctan(0): 0
Loops: 8
arctan(1): 0.724778414
Loops: 13
arctan(R3): 0.709577262
Loops: 6
arctan(1/R3): 0.517280579
R3 is just the square root of 3 which is 1.732050808....
Now I know that the radius of convergence of the arctan series is |x| <= 1, so I'm thinking that I have to reduce the input somehow. The problem is that for arctan, the domain of the function is (-INF, +INF). So how do you reduce that? This is being calculated to radian angles.
Thanks for pointing that out. The problem has been corrected, and I also have the input reduction done as well. Here is the completed and corrected function which now gives the correct answers:
fp32 t32rArcTangent(fp32 number)
{
fp32 a, b, c, d; /* Temp Variables */
fp32 t; /* Number Temp */
uint32 i; /* Loop Counter */
uint8 fr; /* Reduction Flag */
/* Time Savers */
if (b32isInf(number) == -1) return(-(fp32)MM_PI / 2);
if (b32isInf(number) == 1) return((fp32)MM_PI / 2);
if (b32isNaN(number)) return(number);
if (b32fpcomp(number, MM_FP8INFINITY)) return((fp32)MM_PI / 2);
if (b32fpcomp(number, -MM_FP8INFINITY)) return(-(fp32)MM_PI / 2);
if (b32fpcomp(number, ONE)) return((fp32)MM_PI / 4);
if (b32fpcomp(number, -ONE)) return(-(fp32)MM_PI / 4);
/* Reduce Input */
if (number > ONE)
{
number = 1 / number;
fr = 1;
}
else fr = 0;
/* Setup */
a = 0;
b = 0;
c = 1;
d = number;
t = number * number;
/* Calculation Loop */
for (i = 0; i < MMPRVT_FP32_TRIG_LIMIT; i++)
{
b += d / c;
if (b32fpcomp(a, b)) break;
a = b;
c += 2;
d *= -1 * t;
#ifdef DEBUG
printf("a=%g b=%g, c=%g d=%g\n", a, b, c, d);
#endif
}
#ifdef DEBUG
printf("Loops: %lu\n", i);
#endif
/* Result */
if (fr != 0) a = ((fp32)MM_PI / 2) - a;
return(a);
}
Think about what happens to the terms in each loop as a result of the division by c:
c += 2;
d *= -1 * t / c;
First you're dividing by 1 [implicitly, before this], and then by 3, and then by 5, which sounds good, but because you're multiplying d by this term you're effectively dividing by the product of each of the divisors. IOW, instead of
x - 1/3*x^3 + 1/5*x^5 - 1/7*x^7 + 1/9*x^9
which you want, you're computing
x - 1/(1*3)*x^3 + 1/(1*3*5)*x^5 - 1/(1*3*5*7)*x^7 + 1/(1*3*5*7*9)*x^9
You can still use your d *= -t trick, but you should move the division.

fast & efficient least squares fit algorithm in C?

I am trying to implement a linear least squares fit onto 2 arrays of data: time vs amplitude. The only technique I know so far is to test all of the possible m and b points in (y = m*x+b) and then find out which combination fits my data best so that it has the least error. However, I think iterating so many combinations is sometimes useless because it tests out everything. Are there any techniques to speed up the process that I don't know about? Thanks.
Try this code. It fits y = mx + b to your (x,y) data.
The arguments to linreg are
linreg(int n, REAL x[], REAL y[], REAL* b, REAL* m, REAL* r)
n = number of data points
x,y = arrays of data
*b = output intercept
*m = output slope
*r = output correlation coefficient (can be NULL if you don't want it)
The return value is 0 on success, !=0 on failure.
Here's the code
#include "linreg.h"
#include <stdlib.h>
#include <math.h> /* math functions */
//#define REAL float
#define REAL double
inline static REAL sqr(REAL x) {
return x*x;
}
int linreg(int n, const REAL x[], const REAL y[], REAL* m, REAL* b, REAL* r){
REAL sumx = 0.0; /* sum of x */
REAL sumx2 = 0.0; /* sum of x**2 */
REAL sumxy = 0.0; /* sum of x * y */
REAL sumy = 0.0; /* sum of y */
REAL sumy2 = 0.0; /* sum of y**2 */
for (int i=0;i<n;i++){
sumx += x[i];
sumx2 += sqr(x[i]);
sumxy += x[i] * y[i];
sumy += y[i];
sumy2 += sqr(y[i]);
}
REAL denom = (n * sumx2 - sqr(sumx));
if (denom == 0) {
// singular matrix. can't solve the problem.
*m = 0;
*b = 0;
if (r) *r = 0;
return 1;
}
*m = (n * sumxy - sumx * sumy) / denom;
*b = (sumy * sumx2 - sumx * sumxy) / denom;
if (r!=NULL) {
*r = (sumxy - sumx * sumy / n) / /* compute correlation coeff */
sqrt((sumx2 - sqr(sumx)/n) *
(sumy2 - sqr(sumy)/n));
}
return 0;
}
Example
You can run this example online.
int main()
{
int n = 6;
REAL x[6]= {1, 2, 4, 5, 10, 20};
REAL y[6]= {4, 6, 12, 15, 34, 68};
REAL m,b,r;
linreg(n,x,y,&m,&b,&r);
printf("m=%g b=%g r=%g\n",m,b,r);
return 0;
}
Here is the output
m=3.43651 b=-0.888889 r=0.999192
Here is the Excel plot and linear fit (for verification).
All values agree exactly with the C code above (note C code returns r while Excel returns R**2).
There are efficient algorithms for least-squares fitting; see Wikipedia for details. There are also libraries that implement the algorithms for you, likely more efficiently than a naive implementation would do; the GNU Scientific Library is one example, but there are others under more lenient licenses as well.
From Numerical Recipes: The Art of Scientific Computing in (15.2) Fitting Data to a Straight Line:
Linear Regression:
Consider the problem of fitting a set of N data points (xi, yi) to a straight-line model:
Assume that the uncertainty: sigmai associated with each yi and that the xi’s (values of the dependent variable) are known exactly. To measure how well the model agrees with the data, we use the chi-square function, which in this case is:
The above equation is minimized to determine a and b. This is done by finding the derivative of the above equation with respect to a and b, equate them to zero and solve for a and b. Then we estimate the probable uncertainties in the estimates of a and b, since obviously the measurement errors in the data must introduce some uncertainty in the determination of those parameters. Additionally, we must estimate the goodness-of-fit of the data to the
model. Absent this estimate, we have not the slightest indication that the parameters a and b in the model have any meaning at all.
The below struct performs the mentioned calculations:
struct Fitab {
// Object for fitting a straight line y = a + b*x to a set of
// points (xi, yi), with or without available
// errors sigma i . Call one of the two constructors to calculate the fit.
// The answers are then available as the variables:
// a, b, siga, sigb, chi2, and either q or sigdat.
int ndata;
double a, b, siga, sigb, chi2, q, sigdat; // Answers.
vector<double> &x, &y, &sig;
// Constructor.
Fitab(vector<double> &xx, vector<double> &yy, vector<double> &ssig)
: ndata(xx.size()), x(xx), y(yy), sig(ssig), chi2(0.), q(1.), sigdat(0.)
{
// Given a set of data points x[0..ndata-1], y[0..ndata-1]
// with individual standard deviations sig[0..ndata-1],
// sets a,b and their respective probable uncertainties
// siga and sigb, the chi-square: chi2, and the goodness-of-fit
// probability: q
Gamma gam;
int i;
double ss=0., sx=0., sy=0., st2=0., t, wt, sxoss; b=0.0;
for (i=0;i < ndata; i++) { // Accumulate sums ...
wt = 1.0 / SQR(sig[i]); //...with weights
ss += wt;
sx += x[i]*wt;
sy += y[i]*wt;
}
sxoss = sx/ss;
for (i=0; i < ndata; i++) {
t = (x[i]-sxoss) / sig[i];
st2 += t*t;
b += t*y[i]/sig[i];
}
b /= st2; // Solve for a, b, sigma-a, and simga-b.
a = (sy-sx*b) / ss;
siga = sqrt((1.0+sx*sx/(ss*st2))/ss);
sigb = sqrt(1.0/st2); // Calculate chi2.
for (i=0;i<ndata;i++) chi2 += SQR((y[i]-a-b*x[i])/sig[i]);
if (ndata>2) q=gam.gammq(0.5*(ndata-2),0.5*chi2); // goodness of fit
}
// Constructor.
Fitab(vector<double> &xx, vector<double> &yy)
: ndata(xx.size()), x(xx), y(yy), sig(xx), chi2(0.), q(1.), sigdat(0.)
{
// As above, but without known errors (sig is not used).
// The uncertainties siga and sigb are estimated by assuming
// equal errors for all points, and that a straight line is
// a good fit. q is returned as 1.0, the normalization of chi2
// is to unit standard deviation on all points, and sigdat
// is set to the estimated error of each point.
int i;
double ss,sx=0.,sy=0.,st2=0.,t,sxoss;
b=0.0; // Accumulate sums ...
for (i=0; i < ndata; i++) {
sx += x[i]; // ...without weights.
sy += y[i];
}
ss = ndata;
sxoss = sx/ss;
for (i=0;i < ndata; i++) {
t = x[i]-sxoss;
st2 += t*t;
b += t*y[i];
}
b /= st2; // Solve for a, b, sigma-a, and sigma-b.
a = (sy-sx*b)/ss;
siga=sqrt((1.0+sx*sx/(ss*st2))/ss);
sigb=sqrt(1.0/st2); // Calculate chi2.
for (i=0;i<ndata;i++) chi2 += SQR(y[i]-a-b*x[i]);
if (ndata > 2) sigdat=sqrt(chi2/(ndata-2));
// For unweighted data evaluate typical
// sig using chi2, and adjust
// the standard deviations.
siga *= sigdat;
sigb *= sigdat;
}
};
where struct Gamma:
struct Gamma : Gauleg18 {
// Object for incomplete gamma function.
// Gauleg18 provides coefficients for Gauss-Legendre quadrature.
static const Int ASWITCH=100; When to switch to quadrature method.
static const double EPS; // See end of struct for initializations.
static const double FPMIN;
double gln;
double gammp(const double a, const double x) {
// Returns the incomplete gamma function P(a,x)
if (x < 0.0 || a <= 0.0) throw("bad args in gammp");
if (x == 0.0) return 0.0;
else if ((Int)a >= ASWITCH) return gammpapprox(a,x,1); // Quadrature.
else if (x < a+1.0) return gser(a,x); // Use the series representation.
else return 1.0-gcf(a,x); // Use the continued fraction representation.
}
double gammq(const double a, const double x) {
// Returns the incomplete gamma function Q(a,x) = 1 - P(a,x)
if (x < 0.0 || a <= 0.0) throw("bad args in gammq");
if (x == 0.0) return 1.0;
else if ((Int)a >= ASWITCH) return gammpapprox(a,x,0); // Quadrature.
else if (x < a+1.0) return 1.0-gser(a,x); // Use the series representation.
else return gcf(a,x); // Use the continued fraction representation.
}
double gser(const Doub a, const Doub x) {
// Returns the incomplete gamma function P(a,x) evaluated by its series representation.
// Also sets ln (gamma) as gln. User should not call directly.
double sum,del,ap;
gln=gammln(a);
ap=a;
del=sum=1.0/a;
for (;;) {
++ap;
del *= x/ap;
sum += del;
if (fabs(del) < fabs(sum)*EPS) {
return sum*exp(-x+a*log(x)-gln);
}
}
}
double gcf(const Doub a, const Doub x) {
// Returns the incomplete gamma function Q(a, x) evaluated
// by its continued fraction representation.
// Also sets ln (gamma) as gln. User should not call directly.
int i;
double an,b,c,d,del,h;
gln=gammln(a);
b=x+1.0-a; // Set up for evaluating continued fraction
// by modified Lentz’s method with with b0 = 0.
c=1.0/FPMIN;
d=1.0/b;
h=d;
for (i=1;;i++) {
// Iterate to convergence.
an = -i*(i-a);
b += 2.0;
d=an*d+b;
if (fabs(d) < FPMIN) d=FPMIN;
c=b+an/c;
if (fabs(c) < FPMIN) c=FPMIN;
d=1.0/d;
del=d*c;
h *= del;
if (fabs(del-1.0) <= EPS) break;
}
return exp(-x+a*log(x)-gln)*h; Put factors in front.
}
double gammpapprox(double a, double x, int psig) {
// Incomplete gamma by quadrature. Returns P(a,x) or Q(a, x),
// when psig is 1 or 0, respectively. User should not call directly.
int j;
double xu,t,sum,ans;
double a1 = a-1.0, lna1 = log(a1), sqrta1 = sqrt(a1);
gln = gammln(a);
// Set how far to integrate into the tail:
if (x > a1) xu = MAX(a1 + 11.5*sqrta1, x + 6.0*sqrta1);
else xu = MAX(0.,MIN(a1 - 7.5*sqrta1, x - 5.0*sqrta1));
sum = 0;
for (j=0;j<ngau;j++) { // Gauss-Legendre.
t = x + (xu-x)*y[j];
sum += w[j]*exp(-(t-a1)+a1*(log(t)-lna1));
}
ans = sum*(xu-x)*exp(a1*(lna1-1.)-gln);
return (psig?(ans>0.0? 1.0-ans:-ans):(ans>=0.0? ans:1.0+ans));
}
double invgammp(Doub p, Doub a);
// Inverse function on x of P(a,x) .
};
const Doub Gamma::EPS = numeric_limits<Doub>::epsilon();
const Doub Gamma::FPMIN = numeric_limits<Doub>::min()/EPS
and stuct Gauleg18:
struct Gauleg18 {
// Abscissas and weights for Gauss-Legendre quadrature.
static const Int ngau = 18;
static const Doub y[18];
static const Doub w[18];
};
const Doub Gauleg18::y[18] = {0.0021695375159141994,
0.011413521097787704,0.027972308950302116,0.051727015600492421,
0.082502225484340941, 0.12007019910960293,0.16415283300752470,
0.21442376986779355, 0.27051082840644336, 0.33199876341447887,
0.39843234186401943, 0.46931971407375483, 0.54413605556657973,
0.62232745288031077, 0.70331500465597174, 0.78649910768313447,
0.87126389619061517, 0.95698180152629142};
const Doub Gauleg18::w[18] = {0.0055657196642445571,
0.012915947284065419,0.020181515297735382,0.027298621498568734,
0.034213810770299537,0.040875750923643261,0.047235083490265582,
0.053244713977759692,0.058860144245324798,0.064039797355015485
0.068745323835736408,0.072941885005653087,0.076598410645870640,
0.079687828912071670,0.082187266704339706,0.084078218979661945,
0.085346685739338721,0.085983275670394821};
and, finally fuinction Gamma::invgamp():
double Gamma::invgammp(double p, double a) {
// Returns x such that P(a,x) = p for an argument p between 0 and 1.
int j;
double x,err,t,u,pp,lna1,afac,a1=a-1;
const double EPS=1.e-8; // Accuracy is the square of EPS.
gln=gammln(a);
if (a <= 0.) throw("a must be pos in invgammap");
if (p >= 1.) return MAX(100.,a + 100.*sqrt(a));
if (p <= 0.) return 0.0;
if (a > 1.) {
lna1=log(a1);
afac = exp(a1*(lna1-1.)-gln);
pp = (p < 0.5)? p : 1. - p;
t = sqrt(-2.*log(pp));
x = (2.30753+t*0.27061)/(1.+t*(0.99229+t*0.04481)) - t;
if (p < 0.5) x = -x;
x = MAX(1.e-3,a*pow(1.-1./(9.*a)-x/(3.*sqrt(a)),3));
} else {
t = 1.0 - a*(0.253+a*0.12); and (6.2.9).
if (p < t) x = pow(p/t,1./a);
else x = 1.-log(1.-(p-t)/(1.-t));
}
for (j=0;j<12;j++) {
if (x <= 0.0) return 0.0; // x too small to compute accurately.
err = gammp(a,x) - p;
if (a > 1.) t = afac*exp(-(x-a1)+a1*(log(x)-lna1));
else t = exp(-x+a1*log(x)-gln);
u = err/t;
// Halley’s method.
x -= (t = u/(1.-0.5*MIN(1.,u*((a-1.)/x - 1))));
// Halve old value if x tries to go negative.
if (x <= 0.) x = 0.5*(x + t);
if (fabs(t) < EPS*x ) break;
}
return x;
}
Here is my version of a C/C++ function that does simple linear regression. The calculations follow the wikipedia article on simple linear regression. This is published as a single-header public-domain (MIT) library on github: simple_linear_regression. The library (.h file) is tested to work on Linux and Windows, and from C and C++ using -Wall -Werror and all -std versions supported by clang/gcc.
#define SIMPLE_LINEAR_REGRESSION_ERROR_INPUT_VALUE -2
#define SIMPLE_LINEAR_REGRESSION_ERROR_NUMERIC -3
int simple_linear_regression(const double * x, const double * y, const int n, double * slope_out, double * intercept_out, double * r2_out) {
double sum_x = 0.0;
double sum_xx = 0.0;
double sum_xy = 0.0;
double sum_y = 0.0;
double sum_yy = 0.0;
double n_real = (double)(n);
int i = 0;
double slope = 0.0;
double denominator = 0.0;
if (x == NULL || y == NULL || n < 2) {
return SIMPLE_LINEAR_REGRESSION_ERROR_INPUT_VALUE;
}
for (i = 0; i < n; ++i) {
sum_x += x[i];
sum_xx += x[i] * x[i];
sum_xy += x[i] * y[i];
sum_y += y[i];
sum_yy += y[i] * y[i];
}
denominator = n_real * sum_xx - sum_x * sum_x;
if (denominator == 0.0) {
return SIMPLE_LINEAR_REGRESSION_ERROR_NUMERIC;
}
slope = (n_real * sum_xy - sum_x * sum_y) / denominator;
if (slope_out != NULL) {
*slope_out = slope;
}
if (intercept_out != NULL) {
*intercept_out = (sum_y - slope * sum_x) / n_real;
}
if (r2_out != NULL) {
denominator = ((n_real * sum_xx) - (sum_x * sum_x)) * ((n_real * sum_yy) - (sum_y * sum_y));
if (denominator == 0.0) {
return SIMPLE_LINEAR_REGRESSION_ERROR_NUMERIC;
}
*r2_out = ((n_real * sum_xy) - (sum_x * sum_y)) * ((n_real * sum_xy) - (sum_x * sum_y)) / denominator;
}
return 0;
}
Usage example:
#define SIMPLE_LINEAR_REGRESSION_IMPLEMENTATION
#include "simple_linear_regression.h"
#include <stdio.h>
/* Some data that we want to find the slope, intercept and r2 for */
static const double x[] = { 1.47, 1.50, 1.52, 1.55, 1.57, 1.60, 1.63, 1.65, 1.68, 1.70, 1.73, 1.75, 1.78, 1.80, 1.83 };
static const double y[] = { 52.21, 53.12, 54.48, 55.84, 57.20, 58.57, 59.93, 61.29, 63.11, 64.47, 66.28, 68.10, 69.92, 72.19, 74.46 };
int main() {
double slope = 0.0;
double intercept = 0.0;
double r2 = 0.0;
int res = 0;
res = simple_linear_regression(x, y, sizeof(x) / sizeof(x[0]), &slope, &intercept, &r2);
if (res < 0) {
printf("Error: %s\n", simple_linear_regression_error_string(res));
return res;
}
printf("slope: %f\n", slope);
printf("intercept: %f\n", intercept);
printf("r2: %f\n", r2);
return 0;
}
The original example above worked well for me with slope and offset but I had a hard time with the corr coef. Maybe I don't have my parenthesis working the same as the assumed precedence? Anyway, with some help of other web pages I finally got values that match the linear trend-line in Excel. Thought I would share my code using Mark Lakata's variable names. Hope this helps.
double slope = ((n * sumxy) - (sumx * sumy )) / denom;
double intercept = ((sumy * sumx2) - (sumx * sumxy)) / denom;
double term1 = ((n * sumxy) - (sumx * sumy));
double term2 = ((n * sumx2) - (sumx * sumx));
double term3 = ((n * sumy2) - (sumy * sumy));
double term23 = (term2 * term3);
double r2 = 1.0;
if (fabs(term23) > MIN_DOUBLE) // Define MIN_DOUBLE somewhere as 1e-9 or similar
r2 = (term1 * term1) / term23;
as an assignment I had to code in C a simple linear regression using RMSE loss function. The program is dynamic and you can enter your own values and choose your own loss function which is for now limited to Root Mean Square Error. But first here are the algorithms I used:
now the code... you need gnuplot to display the chart, sudo apt install gnuplot
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <math.h>
#include <sys/types.h>
#define BUFFSIZE 64
#define MAXSIZE 100
static double vector_x[MAXSIZE] = {0};
static double vector_y[MAXSIZE] = {0};
static double vector_predict[MAXSIZE] = {0};
static double max_x;
static double max_y;
static double mean_x;
static double mean_y;
static double teta_0_intercept;
static double teta_1_grad;
static double RMSE;
static double r_square;
static double prediction;
static char intercept[BUFFSIZE];
static char grad[BUFFSIZE];
static char xrange[BUFFSIZE];
static char yrange[BUFFSIZE];
static char lossname_RMSE[BUFFSIZE] = "Simple Linear Regression using RMSE'";
static char cmd_gnu_0[BUFFSIZE] = "set title '";
static char cmd_gnu_1[BUFFSIZE] = "intercept = ";
static char cmd_gnu_2[BUFFSIZE] = "grad = ";
static char cmd_gnu_3[BUFFSIZE] = "set xrange [0:";
static char cmd_gnu_4[BUFFSIZE] = "set yrange [0:";
static char cmd_gnu_5[BUFFSIZE] = "f(x) = (grad * x) + intercept";
static char cmd_gnu_6[BUFFSIZE] = "plot f(x), 'data.temp' with points pointtype 7";
static char const *commands_gnuplot[] = {
cmd_gnu_0,
cmd_gnu_1,
cmd_gnu_2,
cmd_gnu_3,
cmd_gnu_4,
cmd_gnu_5,
cmd_gnu_6,
};
static size_t size;
static void user_input()
{
printf("Enter x,y vector size, MAX = 100\n");
scanf("%lu", &size);
if (size > MAXSIZE) {
printf("Wrong input size is too big\n");
user_input();
}
printf("vector's size is %lu\n", size);
size_t i;
for (i = 0; i < size; i++) {
printf("Enter vector_x[%ld] values\n", i);
scanf("%lf", &vector_x[i]);
}
for (i = 0; i < size; i++) {
printf("Enter vector_y[%ld] values\n", i);
scanf("%lf", &vector_y[i]);
}
}
static void display_vector()
{
size_t i;
for (i = 0; i < size; i++){
printf("vector_x[%lu] = %lf\t", i, vector_x[i]);
printf("vector_y[%lu] = %lf\n", i, vector_y[i]);
}
}
static void concatenate(char p[], char q[]) {
int c;
int d;
c = 0;
while (p[c] != '\0') {
c++;
}
d = 0;
while (q[d] != '\0') {
p[c] = q[d];
d++;
c++;
}
p[c] = '\0';
}
static void compute_mean_x_y()
{
size_t i;
double tmp_x = 0.0;
double tmp_y = 0.0;
for (i = 0; i < size; i++) {
tmp_x += vector_x[i];
tmp_y += vector_y[i];
}
mean_x = tmp_x / size;
mean_y = tmp_y / size;
printf("mean_x = %lf\n", mean_x);
printf("mean_y = %lf\n", mean_y);
}
static void compute_teta_1_grad()
{
double numerator = 0.0;
double denominator = 0.0;
double tmp1 = 0.0;
double tmp2 = 0.0;
size_t i;
for (i = 0; i < size; i++) {
numerator += (vector_x[i] - mean_x) * (vector_y[i] - mean_y);
}
for (i = 0; i < size; i++) {
tmp1 = vector_x[i] - mean_x;
tmp2 = tmp1 * tmp1;
denominator += tmp2;
}
teta_1_grad = numerator / denominator;
printf("teta_1_grad = %lf\n", teta_1_grad);
}
static void compute_teta_0_intercept()
{
teta_0_intercept = mean_y - (teta_1_grad * mean_x);
printf("teta_0_intercept = %lf\n", teta_0_intercept);
}
static void compute_prediction()
{
size_t i;
for (i = 0; i < size; i++) {
vector_predict[i] = teta_0_intercept + (teta_1_grad * vector_x[i]);
printf("y^[%ld] = %lf\n", i, vector_predict[i]);
}
printf("\n");
}
static void compute_RMSE()
{
compute_prediction();
double error = 0;
size_t i;
for (i = 0; i < size; i++) {
error = (vector_predict[i] - vector_y[i]) * (vector_predict[i] - vector_y[i]);
printf("error y^[%ld] = %lf\n", i, error);
RMSE += error;
}
/* mean */
RMSE = RMSE / size;
/* square root mean */
RMSE = sqrt(RMSE);
printf("\nRMSE = %lf\n", RMSE);
}
static void compute_loss_function()
{
int input = 0;
printf("Which loss function do you want to use?\n");
printf(" 1 - RMSE\n");
scanf("%d", &input);
switch(input) {
case 1:
concatenate(cmd_gnu_0, lossname_RMSE);
compute_RMSE();
printf("\n");
break;
default:
printf("Wrong input try again\n");
compute_loss_function(size);
}
}
static void compute_r_square(size_t size)
{
double num_err = 0.0;
double den_err = 0.0;
size_t i;
for (i = 0; i < size; i++) {
num_err += (vector_y[i] - vector_predict[i]) * (vector_y[i] - vector_predict[i]);
den_err += (vector_y[i] - mean_y) * (vector_y[i] - mean_y);
}
r_square = 1 - (num_err/den_err);
printf("R_square = %lf\n", r_square);
}
static void compute_predict_for_x()
{
double x = 0.0;
printf("Please enter x value\n");
scanf("%lf", &x);
prediction = teta_0_intercept + (teta_1_grad * x);
printf("y^ if x = %lf -> %lf\n",x, prediction);
}
static void compute_max_x_y()
{
size_t i;
double tmp1= 0.0;
double tmp2= 0.0;
for (i = 0; i < size; i++) {
if (vector_x[i] > tmp1) {
tmp1 = vector_x[i];
max_x = vector_x[i];
}
if (vector_y[i] > tmp2) {
tmp2 = vector_y[i];
max_y = vector_y[i];
}
}
printf("vector_x max value %lf\n", max_x);
printf("vector_y max value %lf\n", max_y);
}
static void display_model_line()
{
sprintf(intercept, "%0.7lf", teta_0_intercept);
sprintf(grad, "%0.7lf", teta_1_grad);
sprintf(xrange, "%0.7lf", max_x + 1);
sprintf(yrange, "%0.7lf", max_y + 1);
concatenate(cmd_gnu_1, intercept);
concatenate(cmd_gnu_2, grad);
concatenate(cmd_gnu_3, xrange);
concatenate(cmd_gnu_3, "]");
concatenate(cmd_gnu_4, yrange);
concatenate(cmd_gnu_4, "]");
printf("grad = %s\n", grad);
printf("intercept = %s\n", intercept);
printf("xrange = %s\n", xrange);
printf("yrange = %s\n", yrange);
printf("cmd_gnu_0: %s\n", cmd_gnu_0);
printf("cmd_gnu_1: %s\n", cmd_gnu_1);
printf("cmd_gnu_2: %s\n", cmd_gnu_2);
printf("cmd_gnu_3: %s\n", cmd_gnu_3);
printf("cmd_gnu_4: %s\n", cmd_gnu_4);
printf("cmd_gnu_5: %s\n", cmd_gnu_5);
printf("cmd_gnu_6: %s\n", cmd_gnu_6);
/* print plot */
FILE *gnuplot_pipe = (FILE*)popen("gnuplot -persistent", "w");
FILE *temp = (FILE*)fopen("data.temp", "w");
/* create data.temp */
size_t i;
for (i = 0; i < size; i++)
{
fprintf(temp, "%f %f \n", vector_x[i], vector_y[i]);
}
/* display gnuplot */
for (i = 0; i < 7; i++)
{
fprintf(gnuplot_pipe, "%s \n", commands_gnuplot[i]);
}
}
int main(void)
{
printf("===========================================\n");
printf("INPUT DATA\n");
printf("===========================================\n");
user_input();
display_vector();
printf("\n");
printf("===========================================\n");
printf("COMPUTE MEAN X:Y, TETA_1 TETA_0\n");
printf("===========================================\n");
compute_mean_x_y();
compute_max_x_y();
compute_teta_1_grad();
compute_teta_0_intercept();
printf("\n");
printf("===========================================\n");
printf("COMPUTE LOSS FUNCTION\n");
printf("===========================================\n");
compute_loss_function();
printf("===========================================\n");
printf("COMPUTE R_square\n");
printf("===========================================\n");
compute_r_square(size);
printf("\n");
printf("===========================================\n");
printf("COMPUTE y^ according to x\n");
printf("===========================================\n");
compute_predict_for_x();
printf("\n");
printf("===========================================\n");
printf("DISPLAY LINEAR REGRESSION\n");
printf("===========================================\n");
display_model_line();
printf("\n");
return 0;
}
Look at Section 1 of this paper. This section expresses a 2D linear regression as a matrix multiplication exercise. As long as your data is well-behaved, this technique should permit you to develop a quick least squares fit.
Depending on the size of your data, it might be worthwhile to algebraically reduce the matrix multiplication to simple set of equations, thereby avoiding the need to write a matmult() function. (Be forewarned, this is completely impractical for more than 4 or 5 data points!)
The fastest, most efficient way to solve least squares, as far as I am aware, is to subtract (the gradient)/(the 2nd order gradient) from your parameter vector. (2nd order gradient = i.e. the diagonal of the Hessian.)
Here is the intuition:
Let's say you want to optimize least squares over a single parameter. This is equivalent to finding the vertex of a parabola. Then, for any random initial parameter, x0, the vertex of the loss function is located at x0 - f(1) / f(2). That's because adding - f(1) / f(2) to x will always zero out the derivative, f(1).
Side note: Implementing this in Tensorflow, the solution appeared at w0 - f(1) / f(2) / (number of weights), but I'm not sure if that's due to Tensorflow or if it's due to something else..

Resources