Poisson calculation (erlang C) - c

i posted this before, user told me to post it on codereview. i did, and they closed it...so one more time here: (i deleted the old question)
I have these formulas:
and I need the Poisson formulas for the erlangC formula:
I tried to rebuild the formulas in C:
double getPoisson(double m, double u, bool cumu)
{
double ret = 0;
if(!cumu)
{
ret = (exp(-u)*pow(u,m)) / (factorial(m));
}
else
{
double facto = 1;
double ehu = exp(-u);
for(int i = 0; i < m; i++)
{
ret = ret + (ehu * pow(u,i)) / facto;
facto *= (i+1);
}
}
return ret;
}
The Erlang C Formula:
double getErlangC(double m, double u, double p)
{
double numerator = getPoisson(m, u, false);
double denominator = getPoisson(m, u, false) + (1-p) * getPoisson(m, u, true);
return numerator/denominator;
}
The main problem is, the m parameter in getPoisson is a big value (>170)
so it wants to calculate >170! but it cannot handle it. I think the primitive data types are too small to work correctly, or what do you say?
BTW: This is the factorial function I use for the first Poisson:
double factorial(double n)
{
if(n >= 1)
return n*factorial(n-1);
else
return 1;
}
Some samples:
Input:
double l = getErlangC(50, 48, 0.96);
printf("%g", l);
Output:
0.694456 (correct)
Input:
double l = getErlangC(100, 96, 0.96);
printf("%g", l);
Output:
0.5872811 (correct)
if i use a value higher than 170 for the first parameter (m) of getErlangC like:
Input:
double l = getErlangC(500, 487, 0.974);
printf("%g", l);
Output:
naN (incorrect)
Excepted:
0.45269
How's my approach? Would be there a better way to calculate Poisson and erlangC?
Some Info: Excel has the POISSON Function, and on Excel it works perfekt... would there be a way to see the algorithm(code) EXCEL uses for POISSON?

(pow(u, m)/factorial(m)) can be expressed as a recursive loop with each element shown as u/n where each n is an element of m!.
double ratio(double u, int n)
{
if(n > 0)
{
// Avoid the ratio overflow by calculating each ratio element
double val;
val = u/n;
return val*ratio(u, n-1);
}
else
{
// Avoid division by 0 as power and factorial of 0 are 1
return 1;
}
}
Note that if you want to avoid recursion, you can do it as a loop as well
double ratio(double u, int n)
{
int i;
// Avoid the ratio overflow by calculating each ratio element
// default the ratio to 1 for n == 0
double val = 1;
// calculate the next n-1 ratios and put them into the total
for (i = 1; i<=n; i++)
{
// Put in the next element of the ratio
val *= u/i;
}
// return the final value of the ratio
return val;
}

To cope with values exceeding the double range, re-code to use the log of values. Downside- some precision loss.
Precision can be re-gained with improved code, but here is something that at least copes with the range issues.
Slight variant of OP's code follows: Used for comparison.
long double factorial(unsigned m) {
long double f = 1.0;
while (m > 0) {
f *= m;
m--;
}
return f;
}
double getPoisson(unsigned m, double u, bool cumu) {
double ret = 0;
if (!cumu) {
ret = (double) ((exp(-u) * pow(u, m)) / (factorial(m)));
} else {
double facto = 1;
double ehu = exp(-u);
for (unsigned i = 0; i < m; i++) {
ret = ret + (ehu * pow(u, i)) / facto;
facto *= (i + 1);
}
}
return ret;
}
double getErlang(unsigned m, double u, double p) {
double numerator = getPoisson(m, u, false);
double denominator = numerator + (1.0 - p) * getPoisson(m, u, true);
return numerator / denominator;
}
Suggested changes
#ifdef M_PI
#define MY_PI M_PI
#else
#define MY_PI 3.1415926535897932384626433832795
#endif
// log of n!
//
// Gosper Approximation of Stirling's Approximation
// http://mathworld.wolfram.com/StirlingsApproximation.html
// n! about= sqrt(pi*(2*n + 1/3.)) * pow(n,n) * exp(-n)
static double ln_factorial(unsigned n) {
if (n <= 1) return 0.0;
double x = n;
return log(sqrt(MY_PI * (2 * x + 1 / 3.0))) + log(x) * x - x;
}
double getPoisson_2(unsigned m, double u, bool cumu) {
double ret = 0.0;
if (cumu) {
// Simplify term calculation. `mul` does not get too large nor small.
double mul = exp(-u);
for (unsigned i = 0; i < m; i++) {
ret += mul;
mul *= u/(i + 1);
// printf("ret:% 10e mul:% 10e\n", ret, mul);
}
} else {
// ret = (exp(-u) * pow(u, m)) / (factorial(m));
double ln_ret = -u + log(u) * m - ln_factorial(m);
return exp(ln_ret);
}
return ret;
}
double getErlang_2(unsigned m, double u, double p) {
double numerator = getPoisson_2(m, u, false);
double denominator = numerator + (1 - p) * getPoisson_2(m, u, true);
return numerator / denominator;
}
Test code
void ErTest(unsigned m, double u, double p, double expect) {
printf("m:%4u u:% 14e p:% 14e", m, u, p);
printf(" E0:% 14e", expect);
double y1 = getErlang(m, u, p);
printf(" E1:% 14e", y1);
double y2 = getErlang_2(m, u, p);
printf(" E2:% 14e", y2);
puts("");
}
int main(void) {
ErTest(50, 48, 0.96, 0.694456);
ErTest(100, 96, 0.96, 0.5872811);
ErTest(500, 487, 0.974, 0.45269);
}
m: 50 u: 4.800000e+01 p: 9.600000e-01 E0: 6.944560e-01 E1: 6.944556e-01 E2: 6.944562e-01
m: 100 u: 9.600000e+01 p: 9.600000e-01 E0: 5.872811e-01 E1: 5.872811e-01 E2: 5.872813e-01
m: 500 u: 4.870000e+02 p: 9.740000e-01 E0: 4.526900e-01 E1: nan E2: 4.464746e-01

Your large recursive factorial is a problem as it might produce a stack overflow as well as a value overflow. pow might also get large.
Here's a way to combine things incrementally:
double
getPoisson(double m, double u, bool cumu)
{
double sum = 0;
double facto = 1;
double u_i = 1;
double ehu = exp(-u);
double cur = ehu;
// u_i -- pow(u,i)
// cur -- current/last term in series
// sum -- sum of terms
for (int i = 0; i < m; i++) {
cur = (ehu * u_i) / facto;
sum += cur;
u_i *= u;
facto *= (i + 1);
}
return cumu ? sum : cur;
}
The above is "okay", but still might overflow some values because of the u_i and facto terms.
Here is an alternate that combines the terms as a ratio. It is less likely to overflow:
double
getPoisson(double m, double u, bool cumu)
{
double sum = 0;
double ehu = exp(-u);
double cur = ehu;
double ratio = 1;
// cur -- current/last term in series
// sum -- sum of terms
// ratio -- u^i / factorial(i)
for (int i = 0; i < m; i++) {
cur = ehu * ratio;
sum += cur;
ratio *= u;
ratio /= (i + 1);
}
return cumu ? sum : cur;
}
The above might still produce some large values. If so, you might have to use long double, quadmath, or multiprecision arithmetic. Or, come up with an "analog" of the equation/algorithm.

Related

Taylor Series in C (problem with sin(240) and sin(300))

#include <stdio.h>
#include <math.h>
const int TERMS = 7;
const float PI = 3.14159265358979;
int fact(int n) {
return n<= 0 ? 1 : n * fact(n-1);
}
double sine(int x) {
double rad = x * (PI / 180);
double sin = 0;
int n;
for(n = 0; n < TERMS; n++) { // That's Taylor series!!
sin += pow(-1, n) * pow(rad, (2 * n) + 1)/ fact((2 * n) + 1);
}
return sin;
}
double cosine(int x) {
double rad = x * (PI / 180);
double cos = 0;
int n;
for(n = 0; n < TERMS; n++) { // That's also Taylor series!
cos += pow(-1, n) * pow(rad, 2 * n) / fact(2 * n);
}
return cos;
}
int main(void){
int y;
scanf("%d",&y);
printf("sine(%d)= %lf\n",y, sine(y));
printf("cosine(%d)= %lf\n",y, cosine(y));
return 0;
}
The code above was implemented to compute sine and cosine using Taylor series.
I tried testing the code and it works fine for sine(120).
I am getting wrong answers for sine(240) and sine(300).
Can anyone help me find out why those errors occur?
You should calculate the functions in the first quadrant only [0, pi/2). Exploit the properties of the functions to get the values for other angles. For instance, for values of x between [pi/2, pi), sin(x) can be calculated by sin(pi - x).
The sine of 120 degrees, which is 40 past 90 degrees, is the same as 50 degrees: 40 degrees before 90. Sine starts at 0, then rises toward 1 at 90 degrees, and then falls again in a mirror image to zero at 180.
The negative sine values from pi to 2pi are just -sin(x - pi). I'd handle everything by this recursive definition:
sin(x):
cases x of:
[0, pi/2) -> calculate (Taylor or whatever)
[pi/2, pi) -> sin(pi - x)
[pi/2, 2pi) -> -sin(x - pi)
< 0 -> sin(-x)
>= 2pi -> sin(fmod(x, 2pi)) // floating-point remainder
A similar approach for cos, using identity cases appropriate for it.
The key point is:
TERMS is too small to have proper precision. And if you increase TERMS, you have to change fact implementation as it will likely overflow when working with int.
I would use a sign to toggle the -1 power instead of pow(-1,n) overkill.
Then use double for the value of PI to avoid losing too many decimals
Then for high values, you should increase the number of terms (this is the main issue). using long long for your factorial method or you get overflow. I set 10 and get proper results:
#include <stdio.h>
#include <math.h>
const int TERMS = 10;
const double PI = 3.14159265358979;
long long fact(int n) {
return n<= 0 ? 1 : n * fact(n-1);
}
double powd(double x,int n) {
return n<= 0 ? 1 : x * powd(x,n-1);
}
double sine(int x) {
double rad = x * (PI / 180);
double sin = 0;
int n;
int sign = 1;
for(n = 0; n < TERMS; n++) { // That's Taylor series!!
sin += sign * powd(rad, (2 * n) + 1)/ fact((2 * n) + 1);
sign = -sign;
}
return sin;
}
double cosine(int x) {
double rad = x * (PI / 180);
double cos = 0;
int n;
int sign = 1;
for(n = 0; n < TERMS; n++) { // That's also Taylor series!
cos += sign * powd(rad, 2 * n) / fact(2 * n);
sign = -sign;
}
return cos;
}
int main(void){
int y;
scanf("%d",&y);
printf("sine(%d)= %lf\n",y, sine(y));
printf("cosine(%d)= %lf\n",y, cosine(y));
return 0;
}
result:
240
sine(240)= -0.866026
cosine(240)= -0.500001
Notes:
my recusive implementation of pow using successive multiplications is probably not needed, since we're dealing with floating point. It introduces accumulation error if n is big.
fact could be using floating point to allow bigger numbers and better precision. Actually I suggested long long but it would be better not to assume that the size will be enough. Better use standard type like int64_t for that.
fact and pow results could be pre-computed/hardcoded as well. This would save computation time.
const double TERMS = 14;
const double PI = 3.14159265358979;
double fact(double n) {return n <= 0.0 ? 1 : n * fact(n - 1);}
double sine(double x)
{
double rad = x * (PI / 180);
rad = fmod(rad, 2 * PI);
double sin = 0;
for (double n = 0; n < TERMS; n++)
sin += pow(-1, n) * pow(rad, (2 * n) + 1) / fact((2 * n) + 1);
return sin;
}
double cosine(double x)
{
double rad = x * (PI / 180);
rad = fmod(rad,2*PI);
double cos = 0;
for (double n = 0; n < TERMS; n++)
cos += pow(-1, n) * pow(rad, 2 * n) / fact(2 * n);
return cos;
}
int main()
{
printf("sine(240)= %lf\n", sine(240));
printf("cosine(300)= %lf\n",cosine(300));
}

Using Heron's formula to calculate square root in C

I have implemented this function:
double heron(double a)
{
double x = (a + 1) / 2;
while (x * x - a > 0.000001) {
x = 0.5 * (x + a / x);
}
return x;
}
This function is working as intended, however I would wish to improve it. It's supposed to use and endless while loop to check if something similar to x * x is a. a is the number the user should input.
So far I have no working function using that method...This is my miserably failed attempt:
double heron(double a)
{
double x = (a + 1) / 2;
while (x * x != a) {
x = 0.5 * (x + a / x);
}
return x;
}
This is my first post so if there is anything unclear or something I should add please let me know.
Failed attempt number 2:
double heron(double a)
{
double x = (a + 1) / 2;
while (1) {
if (x * x == a){
break;
} else {
x = 0.5 * (x + a / x);
}
}
return x;
}
Heron's formula
It's supposed to use and endless while loop to check if something similar to x * x is a
Problems:
Slow convergence
When the initial x is quite wrong, the improved |x - sqrt(a)| error may still be only half as big. Given the wide range of double, the may take hundreds of iterations to get close.
Ref: Heron's formula.
For a novel 1st estimation method: Fast inverse square root.
Overflow
x * x in x * x != a is prone to overflow. x != a/x affords a like test without that range problem. Should overflow occur, x may get "infected" with "infinity" or "not-a-number" and fail to achieve convergence.
Oscillations
Once x is "close" to sqrt(a) (within a factor of 2) , the error convergence is quadratic - the number of bits "right" doubles each iteration. This continues until x == a/x or, due to peculiarities of double math, x will endlessly oscillate between two values as will the quotient.
Getting in this oscillation causes OP's loop to not terminate
Putting this together, with a test harness, demonstrates adequate convergence.
#include <assert.h>
#include <math.h>
#include <stdlib.h>
#include <stdio.h>
double rand_finite_double(void) {
union {
double d;
unsigned char uc[sizeof(double)];
} u;
do {
for (unsigned i = 0; i < sizeof u.uc; i++) {
u.uc[i] = (unsigned char) rand();
}
} while (!isfinite(u.d));
return u.d;
}
double sqrt_heron(double a) {
double x = (a + 1) / 2;
double x_previous = -1.0;
for (int i = 0; i < 1000; i++) {
double quotient = a / x;
if (x == quotient || x == x_previous) {
if (x == quotient) {
return x;
}
return ((x + x_previous) / 2);
}
x_previous = x;
x = 0.5 * (x + quotient);
}
// As this code is (should) never be reached, the `for(i)`
// loop "safety" net code is not needed.
assert(0);
}
double test_heron(double xx) {
double x0 = sqrt(xx);
double x1 = sqrt_heron(xx);
if (x0 != x1) {
double delta = fabs(x1 - x0);
double err = delta / x0;
static double emax = 0.0;
if (err > emax) {
emax = err;
printf(" %-24.17e %-24.17e %-24.17e %-24.17e\n", xx, x0, x1, err);
fflush(stdout);
}
}
return 0;
}
int main(void) {
for (int i = 0; i < 100000000; i++) {
test_heron(fabs(rand_finite_double()));
}
return 0;
}
Improvements
sqrt_heron(0.0) works.
Change code for a better initial guess.
double sqrt_heron(double a) {
if (a > 0.0 && a <= DBL_MAX) {
// Better initial guess - halve the exponent of `a`
// Could possible use bit inspection if `double` format known.
int expo;
double significand = frexp(a, &expo);
double x = ldexp(significand, expo / 2);
double x_previous = -1.0;
for (int i = 0; i < 8; i++) { // Notice limit moved from 1000 down to < 10
double quotient = a / x;
if (x == quotient) {
return x;
}
if (x == x_previous) {
return (0.5 * (x + x_previous));
}
x_previous = x;
x = 0.5 * (x + quotient);
}
assert(0);
}
if (a >= 0.0) return a;
assert(0); // invalid argument.
}

C programming - combining power(x, n) and fact(n) functions together

I've been working on a program that calculates sin(x), cos(x), and exp(x) without using math.h and compares them to the library values of their functions. I've been forbidden from actually using the basic power(x, n) and fact(n) functions. The only hint is that I have to do division before doing multiplication when combining the functions into one.
double power(double x, int n)
{
int i;
double prod=1.;
for(i=0;i<n;i++){
prod = prod*x;
}
return prod;
}
double fact(int n)
{
int i;
double prod=1.;
for(i=1;i<=n;i++) {
prod = prod*i;
}
return prod;
}
My idea is to somehow nest the for-loops together, and piecemeal the Taylor Expansion formula for each iteration of the loop, but I haven't had luck actually combining the two.
Any help or hint would be appreciated on how to combine these.
The other aspect of the program that confuses me is that there can only be a single input of X per iteration of the program, and therefore no dynamically defined 'n' for the loops.
Use Taylor series for the exponential:
e^x = 1 + x/1! + x^2/2! + x^3/3!...
and using Euler after that you can calculate sinx and cosx.
The trick is to look at the changes between each successive term in the Taylor series expansion. Let's start with ex:
e^x = 1 + x + x^2/2! + x^3/3! + x^4/4! + x^5/5! ...
Notice that each term is x / n times the prior term, where n is the term number. So start with a term of 1, then multiply by the above expression to get the next term.
That gives you the following implementation:
double etox(double x)
{
long double sum = 0;
// term starts at 1
long double term = 1;
// term number
int i = 1;
// continue until the term is below the precision of the current sum
while (sum + term != sum) {
sum += term;
// new term is x/i times the prior term, where i is the term number
term *= (long double)x / i;
i++;
}
return sum;
}
Note that with this implementation, you'll get some degree of error in the least significant digits. If you start adding from a higher term number and work your way back, this can be avoided.
Similarly for sin(x) and cos(x):
sin(x) = x - x^3/3! + x^5/5! - x^7/7! + x^9/9! ...
cos(x) = 1 - x^2/2! + x^4/4! - x^6/6! + x^8/8! ...
Each term is - (x*x) / ((2*n)*((2*n)-1)) times the prior term, where n is the term number.
I'll leave the the implementation of these two as an exercise for the reader.
Part of this comes from my answer for doing this in MIPS assembly: Taylor Series in MIPS assembly
You can do Taylor series on the fly without having to call sub-functions. In the series, each term can be calculated from the previous term in a loop. (i.e. no need to call fact and/or pow repeatedly, where each starts from the beginning). See https://en.wikipedia.org/wiki/Taylor_series
Anyway, here's code for sin and cos:
// mipstaylor/mipstaylor -- fast sine/cosine calculation
#include <stdio.h>
#include <math.h>
#define ITERMAX 10
// qcos -- calculate cosine
double
qcos(double x)
{
int iteridx;
double x2;
double cur;
int neg;
double xpow;
double n2m1;
double nfac;
double sum;
// square of x
x2 = x * x;
// values for initial terms where n==0:
xpow = 1.0;
n2m1 = 0.0;
nfac = 1.0;
neg = 1;
sum = 0.0;
iteridx = 0;
// NOTES:
// (1) with the setup above, we can just use the loop without any special
// casing
while (1) {
// calculate current value
cur = xpow / nfac;
// apply it to sum
if (neg < 0)
sum -= cur;
else
sum += cur;
// bug out when done
if (++iteridx >= ITERMAX)
break;
// now calculate intermediate values for _next_ sum term
// get _next_ power term
xpow *= x2;
// go from factorial(2n) to factorial(2n+1)
n2m1 += 1.0;
nfac *= n2m1;
// now get factorial(2n+1+1)
n2m1 += 1.0;
nfac *= n2m1;
// flip sign
neg = -neg;
}
return sum;
}
// qsin -- calculate sine
double
qsin(double x)
{
int iteridx;
double x2;
double cur;
int neg;
double xpow;
double n2m1;
double nfac;
double sum;
// square of x
x2 = x * x;
// values for initial terms where n==0:
xpow = x;
n2m1 = 1.0;
nfac = 1.0;
neg = 1;
sum = 0.0;
iteridx = 0;
// NOTES:
// (1) with the setup above, we can just use the loop without any special
// casing
while (1) {
// calculate current value
cur = xpow / nfac;
// apply it to sum
if (neg < 0)
sum -= cur;
else
sum += cur;
// bug out when done
if (++iteridx >= ITERMAX)
break;
// now calculate intermediate values for _next_ sum term
// get _next_ power term
xpow *= x2;
// go from factorial(2n+1) to factorial(2n+1+1)
n2m1 += 1.0;
nfac *= n2m1;
// now get factorial(2n+1+1+1)
n2m1 += 1.0;
nfac *= n2m1;
// flip sign
neg = -neg;
}
return sum;
}
// testfnc -- test function
void
testfnc(int typ,const char *sym)
{
double (*efnc)(double);
double (*qfnc)(double);
double vale;
double valq;
double x;
double dif;
int iter;
switch (typ) {
case 0:
efnc = cos;
qfnc = qcos;
break;
case 1:
efnc = sin;
qfnc = qsin;
break;
default:
efnc = NULL;
qfnc = NULL;
break;
}
iter = 0;
for (x = 0.0; x <= M_PI_2; x += 0.001, ++iter) {
vale = efnc(x);
valq = qfnc(x);
dif = vale - valq;
dif = fabs(dif);
printf("%s: %d x=%.15f e=%.15f q=%.15f dif=%.15f %s\n",
sym,iter,x,vale,valq,dif,(dif < 1e-14) ? "PASS" : "FAIL");
}
}
// main -- main program
int
main(int argc,char **argv)
{
testfnc(0,"cos");
testfnc(1,"sin");
return 0;
}

Realtime Band-Limited Impulse Train Synthesis using SDL mixer

I'm trying to implement a audio synthesizer using this technique:
https://ccrma.stanford.edu/~stilti/papers/blit.pdf
I'm doing it in standard C, using SDL2_Mixer library.
This is my BLIT function implementation:
double blit(double angle, double M, double P) {
double x = M * angle / P;
double denom = (M * sin(M_PI * angle / P));
if (denom < 1)
return (M / P) * cos(M_PI * x) / cos(M_PI * x / M);
else {
double numerator = sin(M_PI * x);
return (M / P) * numerator / denom;
}
}
The idea is to combine it to generate a square wave, following the paper instructions. I setted up SDL2_mixer with this configuration:
SDL_AudioSpec *desired, *obtained;
SDL_AudioSpec *hardware_spec;
desired = (SDL_AudioSpec*)malloc(sizeof(SDL_AudioSpec));
obtained = (SDL_AudioSpec*)malloc(sizeof(SDL_AudioSpec));
desired->freq=44100;
desired->format=AUDIO_U8;
desired->channels=1;
desired->samples=2048;
desired->callback=create_rect;
desired->userdata=NULL;
And here's my create_rect function. It creates a bipolar impulse train, then it integrates it's value to generate a band-limited rect function.
void create_rect(void *userdata, Uint8 *stream, int len) {
static double angle = 0;
static double integral = 0;
int i = 0;
// This is the freq of my tone
double f1 = tone_table[current_wave.note];
// Sample rate
double fs = 44100;
// Pulse
double P = fs / f1;
int M = 2 * floor(P / 2) + 1;
double oldbipolar = 0;
double bipolar = 0;
for(i = 0; i < len; i++) {
if (++angle > P)
angle -= P;
double angle2 = angle + floor(P/2);
if (angle2 > P)
angle2 -= P;
bipolar = blit(angle2, M, P) - blit(angle, M, P);
integral += (bipolar + old bipolar) * 0.5;
oldbipolar = bipolar;
*stream++ = (integral + 0.5) * 127;
}
}
My problem is: the resulting wave is quite ok, but after few seconds it starts to make noises. I tried to plot the result, and here's it:
Any idea?
EDIT: Here's a plot of the bipolar BLIT before integrating it:

fast & efficient least squares fit algorithm in C?

I am trying to implement a linear least squares fit onto 2 arrays of data: time vs amplitude. The only technique I know so far is to test all of the possible m and b points in (y = m*x+b) and then find out which combination fits my data best so that it has the least error. However, I think iterating so many combinations is sometimes useless because it tests out everything. Are there any techniques to speed up the process that I don't know about? Thanks.
Try this code. It fits y = mx + b to your (x,y) data.
The arguments to linreg are
linreg(int n, REAL x[], REAL y[], REAL* b, REAL* m, REAL* r)
n = number of data points
x,y = arrays of data
*b = output intercept
*m = output slope
*r = output correlation coefficient (can be NULL if you don't want it)
The return value is 0 on success, !=0 on failure.
Here's the code
#include "linreg.h"
#include <stdlib.h>
#include <math.h> /* math functions */
//#define REAL float
#define REAL double
inline static REAL sqr(REAL x) {
return x*x;
}
int linreg(int n, const REAL x[], const REAL y[], REAL* m, REAL* b, REAL* r){
REAL sumx = 0.0; /* sum of x */
REAL sumx2 = 0.0; /* sum of x**2 */
REAL sumxy = 0.0; /* sum of x * y */
REAL sumy = 0.0; /* sum of y */
REAL sumy2 = 0.0; /* sum of y**2 */
for (int i=0;i<n;i++){
sumx += x[i];
sumx2 += sqr(x[i]);
sumxy += x[i] * y[i];
sumy += y[i];
sumy2 += sqr(y[i]);
}
REAL denom = (n * sumx2 - sqr(sumx));
if (denom == 0) {
// singular matrix. can't solve the problem.
*m = 0;
*b = 0;
if (r) *r = 0;
return 1;
}
*m = (n * sumxy - sumx * sumy) / denom;
*b = (sumy * sumx2 - sumx * sumxy) / denom;
if (r!=NULL) {
*r = (sumxy - sumx * sumy / n) / /* compute correlation coeff */
sqrt((sumx2 - sqr(sumx)/n) *
(sumy2 - sqr(sumy)/n));
}
return 0;
}
Example
You can run this example online.
int main()
{
int n = 6;
REAL x[6]= {1, 2, 4, 5, 10, 20};
REAL y[6]= {4, 6, 12, 15, 34, 68};
REAL m,b,r;
linreg(n,x,y,&m,&b,&r);
printf("m=%g b=%g r=%g\n",m,b,r);
return 0;
}
Here is the output
m=3.43651 b=-0.888889 r=0.999192
Here is the Excel plot and linear fit (for verification).
All values agree exactly with the C code above (note C code returns r while Excel returns R**2).
There are efficient algorithms for least-squares fitting; see Wikipedia for details. There are also libraries that implement the algorithms for you, likely more efficiently than a naive implementation would do; the GNU Scientific Library is one example, but there are others under more lenient licenses as well.
From Numerical Recipes: The Art of Scientific Computing in (15.2) Fitting Data to a Straight Line:
Linear Regression:
Consider the problem of fitting a set of N data points (xi, yi) to a straight-line model:
Assume that the uncertainty: sigmai associated with each yi and that the xi’s (values of the dependent variable) are known exactly. To measure how well the model agrees with the data, we use the chi-square function, which in this case is:
The above equation is minimized to determine a and b. This is done by finding the derivative of the above equation with respect to a and b, equate them to zero and solve for a and b. Then we estimate the probable uncertainties in the estimates of a and b, since obviously the measurement errors in the data must introduce some uncertainty in the determination of those parameters. Additionally, we must estimate the goodness-of-fit of the data to the
model. Absent this estimate, we have not the slightest indication that the parameters a and b in the model have any meaning at all.
The below struct performs the mentioned calculations:
struct Fitab {
// Object for fitting a straight line y = a + b*x to a set of
// points (xi, yi), with or without available
// errors sigma i . Call one of the two constructors to calculate the fit.
// The answers are then available as the variables:
// a, b, siga, sigb, chi2, and either q or sigdat.
int ndata;
double a, b, siga, sigb, chi2, q, sigdat; // Answers.
vector<double> &x, &y, &sig;
// Constructor.
Fitab(vector<double> &xx, vector<double> &yy, vector<double> &ssig)
: ndata(xx.size()), x(xx), y(yy), sig(ssig), chi2(0.), q(1.), sigdat(0.)
{
// Given a set of data points x[0..ndata-1], y[0..ndata-1]
// with individual standard deviations sig[0..ndata-1],
// sets a,b and their respective probable uncertainties
// siga and sigb, the chi-square: chi2, and the goodness-of-fit
// probability: q
Gamma gam;
int i;
double ss=0., sx=0., sy=0., st2=0., t, wt, sxoss; b=0.0;
for (i=0;i < ndata; i++) { // Accumulate sums ...
wt = 1.0 / SQR(sig[i]); //...with weights
ss += wt;
sx += x[i]*wt;
sy += y[i]*wt;
}
sxoss = sx/ss;
for (i=0; i < ndata; i++) {
t = (x[i]-sxoss) / sig[i];
st2 += t*t;
b += t*y[i]/sig[i];
}
b /= st2; // Solve for a, b, sigma-a, and simga-b.
a = (sy-sx*b) / ss;
siga = sqrt((1.0+sx*sx/(ss*st2))/ss);
sigb = sqrt(1.0/st2); // Calculate chi2.
for (i=0;i<ndata;i++) chi2 += SQR((y[i]-a-b*x[i])/sig[i]);
if (ndata>2) q=gam.gammq(0.5*(ndata-2),0.5*chi2); // goodness of fit
}
// Constructor.
Fitab(vector<double> &xx, vector<double> &yy)
: ndata(xx.size()), x(xx), y(yy), sig(xx), chi2(0.), q(1.), sigdat(0.)
{
// As above, but without known errors (sig is not used).
// The uncertainties siga and sigb are estimated by assuming
// equal errors for all points, and that a straight line is
// a good fit. q is returned as 1.0, the normalization of chi2
// is to unit standard deviation on all points, and sigdat
// is set to the estimated error of each point.
int i;
double ss,sx=0.,sy=0.,st2=0.,t,sxoss;
b=0.0; // Accumulate sums ...
for (i=0; i < ndata; i++) {
sx += x[i]; // ...without weights.
sy += y[i];
}
ss = ndata;
sxoss = sx/ss;
for (i=0;i < ndata; i++) {
t = x[i]-sxoss;
st2 += t*t;
b += t*y[i];
}
b /= st2; // Solve for a, b, sigma-a, and sigma-b.
a = (sy-sx*b)/ss;
siga=sqrt((1.0+sx*sx/(ss*st2))/ss);
sigb=sqrt(1.0/st2); // Calculate chi2.
for (i=0;i<ndata;i++) chi2 += SQR(y[i]-a-b*x[i]);
if (ndata > 2) sigdat=sqrt(chi2/(ndata-2));
// For unweighted data evaluate typical
// sig using chi2, and adjust
// the standard deviations.
siga *= sigdat;
sigb *= sigdat;
}
};
where struct Gamma:
struct Gamma : Gauleg18 {
// Object for incomplete gamma function.
// Gauleg18 provides coefficients for Gauss-Legendre quadrature.
static const Int ASWITCH=100; When to switch to quadrature method.
static const double EPS; // See end of struct for initializations.
static const double FPMIN;
double gln;
double gammp(const double a, const double x) {
// Returns the incomplete gamma function P(a,x)
if (x < 0.0 || a <= 0.0) throw("bad args in gammp");
if (x == 0.0) return 0.0;
else if ((Int)a >= ASWITCH) return gammpapprox(a,x,1); // Quadrature.
else if (x < a+1.0) return gser(a,x); // Use the series representation.
else return 1.0-gcf(a,x); // Use the continued fraction representation.
}
double gammq(const double a, const double x) {
// Returns the incomplete gamma function Q(a,x) = 1 - P(a,x)
if (x < 0.0 || a <= 0.0) throw("bad args in gammq");
if (x == 0.0) return 1.0;
else if ((Int)a >= ASWITCH) return gammpapprox(a,x,0); // Quadrature.
else if (x < a+1.0) return 1.0-gser(a,x); // Use the series representation.
else return gcf(a,x); // Use the continued fraction representation.
}
double gser(const Doub a, const Doub x) {
// Returns the incomplete gamma function P(a,x) evaluated by its series representation.
// Also sets ln (gamma) as gln. User should not call directly.
double sum,del,ap;
gln=gammln(a);
ap=a;
del=sum=1.0/a;
for (;;) {
++ap;
del *= x/ap;
sum += del;
if (fabs(del) < fabs(sum)*EPS) {
return sum*exp(-x+a*log(x)-gln);
}
}
}
double gcf(const Doub a, const Doub x) {
// Returns the incomplete gamma function Q(a, x) evaluated
// by its continued fraction representation.
// Also sets ln (gamma) as gln. User should not call directly.
int i;
double an,b,c,d,del,h;
gln=gammln(a);
b=x+1.0-a; // Set up for evaluating continued fraction
// by modified Lentz’s method with with b0 = 0.
c=1.0/FPMIN;
d=1.0/b;
h=d;
for (i=1;;i++) {
// Iterate to convergence.
an = -i*(i-a);
b += 2.0;
d=an*d+b;
if (fabs(d) < FPMIN) d=FPMIN;
c=b+an/c;
if (fabs(c) < FPMIN) c=FPMIN;
d=1.0/d;
del=d*c;
h *= del;
if (fabs(del-1.0) <= EPS) break;
}
return exp(-x+a*log(x)-gln)*h; Put factors in front.
}
double gammpapprox(double a, double x, int psig) {
// Incomplete gamma by quadrature. Returns P(a,x) or Q(a, x),
// when psig is 1 or 0, respectively. User should not call directly.
int j;
double xu,t,sum,ans;
double a1 = a-1.0, lna1 = log(a1), sqrta1 = sqrt(a1);
gln = gammln(a);
// Set how far to integrate into the tail:
if (x > a1) xu = MAX(a1 + 11.5*sqrta1, x + 6.0*sqrta1);
else xu = MAX(0.,MIN(a1 - 7.5*sqrta1, x - 5.0*sqrta1));
sum = 0;
for (j=0;j<ngau;j++) { // Gauss-Legendre.
t = x + (xu-x)*y[j];
sum += w[j]*exp(-(t-a1)+a1*(log(t)-lna1));
}
ans = sum*(xu-x)*exp(a1*(lna1-1.)-gln);
return (psig?(ans>0.0? 1.0-ans:-ans):(ans>=0.0? ans:1.0+ans));
}
double invgammp(Doub p, Doub a);
// Inverse function on x of P(a,x) .
};
const Doub Gamma::EPS = numeric_limits<Doub>::epsilon();
const Doub Gamma::FPMIN = numeric_limits<Doub>::min()/EPS
and stuct Gauleg18:
struct Gauleg18 {
// Abscissas and weights for Gauss-Legendre quadrature.
static const Int ngau = 18;
static const Doub y[18];
static const Doub w[18];
};
const Doub Gauleg18::y[18] = {0.0021695375159141994,
0.011413521097787704,0.027972308950302116,0.051727015600492421,
0.082502225484340941, 0.12007019910960293,0.16415283300752470,
0.21442376986779355, 0.27051082840644336, 0.33199876341447887,
0.39843234186401943, 0.46931971407375483, 0.54413605556657973,
0.62232745288031077, 0.70331500465597174, 0.78649910768313447,
0.87126389619061517, 0.95698180152629142};
const Doub Gauleg18::w[18] = {0.0055657196642445571,
0.012915947284065419,0.020181515297735382,0.027298621498568734,
0.034213810770299537,0.040875750923643261,0.047235083490265582,
0.053244713977759692,0.058860144245324798,0.064039797355015485
0.068745323835736408,0.072941885005653087,0.076598410645870640,
0.079687828912071670,0.082187266704339706,0.084078218979661945,
0.085346685739338721,0.085983275670394821};
and, finally fuinction Gamma::invgamp():
double Gamma::invgammp(double p, double a) {
// Returns x such that P(a,x) = p for an argument p between 0 and 1.
int j;
double x,err,t,u,pp,lna1,afac,a1=a-1;
const double EPS=1.e-8; // Accuracy is the square of EPS.
gln=gammln(a);
if (a <= 0.) throw("a must be pos in invgammap");
if (p >= 1.) return MAX(100.,a + 100.*sqrt(a));
if (p <= 0.) return 0.0;
if (a > 1.) {
lna1=log(a1);
afac = exp(a1*(lna1-1.)-gln);
pp = (p < 0.5)? p : 1. - p;
t = sqrt(-2.*log(pp));
x = (2.30753+t*0.27061)/(1.+t*(0.99229+t*0.04481)) - t;
if (p < 0.5) x = -x;
x = MAX(1.e-3,a*pow(1.-1./(9.*a)-x/(3.*sqrt(a)),3));
} else {
t = 1.0 - a*(0.253+a*0.12); and (6.2.9).
if (p < t) x = pow(p/t,1./a);
else x = 1.-log(1.-(p-t)/(1.-t));
}
for (j=0;j<12;j++) {
if (x <= 0.0) return 0.0; // x too small to compute accurately.
err = gammp(a,x) - p;
if (a > 1.) t = afac*exp(-(x-a1)+a1*(log(x)-lna1));
else t = exp(-x+a1*log(x)-gln);
u = err/t;
// Halley’s method.
x -= (t = u/(1.-0.5*MIN(1.,u*((a-1.)/x - 1))));
// Halve old value if x tries to go negative.
if (x <= 0.) x = 0.5*(x + t);
if (fabs(t) < EPS*x ) break;
}
return x;
}
Here is my version of a C/C++ function that does simple linear regression. The calculations follow the wikipedia article on simple linear regression. This is published as a single-header public-domain (MIT) library on github: simple_linear_regression. The library (.h file) is tested to work on Linux and Windows, and from C and C++ using -Wall -Werror and all -std versions supported by clang/gcc.
#define SIMPLE_LINEAR_REGRESSION_ERROR_INPUT_VALUE -2
#define SIMPLE_LINEAR_REGRESSION_ERROR_NUMERIC -3
int simple_linear_regression(const double * x, const double * y, const int n, double * slope_out, double * intercept_out, double * r2_out) {
double sum_x = 0.0;
double sum_xx = 0.0;
double sum_xy = 0.0;
double sum_y = 0.0;
double sum_yy = 0.0;
double n_real = (double)(n);
int i = 0;
double slope = 0.0;
double denominator = 0.0;
if (x == NULL || y == NULL || n < 2) {
return SIMPLE_LINEAR_REGRESSION_ERROR_INPUT_VALUE;
}
for (i = 0; i < n; ++i) {
sum_x += x[i];
sum_xx += x[i] * x[i];
sum_xy += x[i] * y[i];
sum_y += y[i];
sum_yy += y[i] * y[i];
}
denominator = n_real * sum_xx - sum_x * sum_x;
if (denominator == 0.0) {
return SIMPLE_LINEAR_REGRESSION_ERROR_NUMERIC;
}
slope = (n_real * sum_xy - sum_x * sum_y) / denominator;
if (slope_out != NULL) {
*slope_out = slope;
}
if (intercept_out != NULL) {
*intercept_out = (sum_y - slope * sum_x) / n_real;
}
if (r2_out != NULL) {
denominator = ((n_real * sum_xx) - (sum_x * sum_x)) * ((n_real * sum_yy) - (sum_y * sum_y));
if (denominator == 0.0) {
return SIMPLE_LINEAR_REGRESSION_ERROR_NUMERIC;
}
*r2_out = ((n_real * sum_xy) - (sum_x * sum_y)) * ((n_real * sum_xy) - (sum_x * sum_y)) / denominator;
}
return 0;
}
Usage example:
#define SIMPLE_LINEAR_REGRESSION_IMPLEMENTATION
#include "simple_linear_regression.h"
#include <stdio.h>
/* Some data that we want to find the slope, intercept and r2 for */
static const double x[] = { 1.47, 1.50, 1.52, 1.55, 1.57, 1.60, 1.63, 1.65, 1.68, 1.70, 1.73, 1.75, 1.78, 1.80, 1.83 };
static const double y[] = { 52.21, 53.12, 54.48, 55.84, 57.20, 58.57, 59.93, 61.29, 63.11, 64.47, 66.28, 68.10, 69.92, 72.19, 74.46 };
int main() {
double slope = 0.0;
double intercept = 0.0;
double r2 = 0.0;
int res = 0;
res = simple_linear_regression(x, y, sizeof(x) / sizeof(x[0]), &slope, &intercept, &r2);
if (res < 0) {
printf("Error: %s\n", simple_linear_regression_error_string(res));
return res;
}
printf("slope: %f\n", slope);
printf("intercept: %f\n", intercept);
printf("r2: %f\n", r2);
return 0;
}
The original example above worked well for me with slope and offset but I had a hard time with the corr coef. Maybe I don't have my parenthesis working the same as the assumed precedence? Anyway, with some help of other web pages I finally got values that match the linear trend-line in Excel. Thought I would share my code using Mark Lakata's variable names. Hope this helps.
double slope = ((n * sumxy) - (sumx * sumy )) / denom;
double intercept = ((sumy * sumx2) - (sumx * sumxy)) / denom;
double term1 = ((n * sumxy) - (sumx * sumy));
double term2 = ((n * sumx2) - (sumx * sumx));
double term3 = ((n * sumy2) - (sumy * sumy));
double term23 = (term2 * term3);
double r2 = 1.0;
if (fabs(term23) > MIN_DOUBLE) // Define MIN_DOUBLE somewhere as 1e-9 or similar
r2 = (term1 * term1) / term23;
as an assignment I had to code in C a simple linear regression using RMSE loss function. The program is dynamic and you can enter your own values and choose your own loss function which is for now limited to Root Mean Square Error. But first here are the algorithms I used:
now the code... you need gnuplot to display the chart, sudo apt install gnuplot
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <math.h>
#include <sys/types.h>
#define BUFFSIZE 64
#define MAXSIZE 100
static double vector_x[MAXSIZE] = {0};
static double vector_y[MAXSIZE] = {0};
static double vector_predict[MAXSIZE] = {0};
static double max_x;
static double max_y;
static double mean_x;
static double mean_y;
static double teta_0_intercept;
static double teta_1_grad;
static double RMSE;
static double r_square;
static double prediction;
static char intercept[BUFFSIZE];
static char grad[BUFFSIZE];
static char xrange[BUFFSIZE];
static char yrange[BUFFSIZE];
static char lossname_RMSE[BUFFSIZE] = "Simple Linear Regression using RMSE'";
static char cmd_gnu_0[BUFFSIZE] = "set title '";
static char cmd_gnu_1[BUFFSIZE] = "intercept = ";
static char cmd_gnu_2[BUFFSIZE] = "grad = ";
static char cmd_gnu_3[BUFFSIZE] = "set xrange [0:";
static char cmd_gnu_4[BUFFSIZE] = "set yrange [0:";
static char cmd_gnu_5[BUFFSIZE] = "f(x) = (grad * x) + intercept";
static char cmd_gnu_6[BUFFSIZE] = "plot f(x), 'data.temp' with points pointtype 7";
static char const *commands_gnuplot[] = {
cmd_gnu_0,
cmd_gnu_1,
cmd_gnu_2,
cmd_gnu_3,
cmd_gnu_4,
cmd_gnu_5,
cmd_gnu_6,
};
static size_t size;
static void user_input()
{
printf("Enter x,y vector size, MAX = 100\n");
scanf("%lu", &size);
if (size > MAXSIZE) {
printf("Wrong input size is too big\n");
user_input();
}
printf("vector's size is %lu\n", size);
size_t i;
for (i = 0; i < size; i++) {
printf("Enter vector_x[%ld] values\n", i);
scanf("%lf", &vector_x[i]);
}
for (i = 0; i < size; i++) {
printf("Enter vector_y[%ld] values\n", i);
scanf("%lf", &vector_y[i]);
}
}
static void display_vector()
{
size_t i;
for (i = 0; i < size; i++){
printf("vector_x[%lu] = %lf\t", i, vector_x[i]);
printf("vector_y[%lu] = %lf\n", i, vector_y[i]);
}
}
static void concatenate(char p[], char q[]) {
int c;
int d;
c = 0;
while (p[c] != '\0') {
c++;
}
d = 0;
while (q[d] != '\0') {
p[c] = q[d];
d++;
c++;
}
p[c] = '\0';
}
static void compute_mean_x_y()
{
size_t i;
double tmp_x = 0.0;
double tmp_y = 0.0;
for (i = 0; i < size; i++) {
tmp_x += vector_x[i];
tmp_y += vector_y[i];
}
mean_x = tmp_x / size;
mean_y = tmp_y / size;
printf("mean_x = %lf\n", mean_x);
printf("mean_y = %lf\n", mean_y);
}
static void compute_teta_1_grad()
{
double numerator = 0.0;
double denominator = 0.0;
double tmp1 = 0.0;
double tmp2 = 0.0;
size_t i;
for (i = 0; i < size; i++) {
numerator += (vector_x[i] - mean_x) * (vector_y[i] - mean_y);
}
for (i = 0; i < size; i++) {
tmp1 = vector_x[i] - mean_x;
tmp2 = tmp1 * tmp1;
denominator += tmp2;
}
teta_1_grad = numerator / denominator;
printf("teta_1_grad = %lf\n", teta_1_grad);
}
static void compute_teta_0_intercept()
{
teta_0_intercept = mean_y - (teta_1_grad * mean_x);
printf("teta_0_intercept = %lf\n", teta_0_intercept);
}
static void compute_prediction()
{
size_t i;
for (i = 0; i < size; i++) {
vector_predict[i] = teta_0_intercept + (teta_1_grad * vector_x[i]);
printf("y^[%ld] = %lf\n", i, vector_predict[i]);
}
printf("\n");
}
static void compute_RMSE()
{
compute_prediction();
double error = 0;
size_t i;
for (i = 0; i < size; i++) {
error = (vector_predict[i] - vector_y[i]) * (vector_predict[i] - vector_y[i]);
printf("error y^[%ld] = %lf\n", i, error);
RMSE += error;
}
/* mean */
RMSE = RMSE / size;
/* square root mean */
RMSE = sqrt(RMSE);
printf("\nRMSE = %lf\n", RMSE);
}
static void compute_loss_function()
{
int input = 0;
printf("Which loss function do you want to use?\n");
printf(" 1 - RMSE\n");
scanf("%d", &input);
switch(input) {
case 1:
concatenate(cmd_gnu_0, lossname_RMSE);
compute_RMSE();
printf("\n");
break;
default:
printf("Wrong input try again\n");
compute_loss_function(size);
}
}
static void compute_r_square(size_t size)
{
double num_err = 0.0;
double den_err = 0.0;
size_t i;
for (i = 0; i < size; i++) {
num_err += (vector_y[i] - vector_predict[i]) * (vector_y[i] - vector_predict[i]);
den_err += (vector_y[i] - mean_y) * (vector_y[i] - mean_y);
}
r_square = 1 - (num_err/den_err);
printf("R_square = %lf\n", r_square);
}
static void compute_predict_for_x()
{
double x = 0.0;
printf("Please enter x value\n");
scanf("%lf", &x);
prediction = teta_0_intercept + (teta_1_grad * x);
printf("y^ if x = %lf -> %lf\n",x, prediction);
}
static void compute_max_x_y()
{
size_t i;
double tmp1= 0.0;
double tmp2= 0.0;
for (i = 0; i < size; i++) {
if (vector_x[i] > tmp1) {
tmp1 = vector_x[i];
max_x = vector_x[i];
}
if (vector_y[i] > tmp2) {
tmp2 = vector_y[i];
max_y = vector_y[i];
}
}
printf("vector_x max value %lf\n", max_x);
printf("vector_y max value %lf\n", max_y);
}
static void display_model_line()
{
sprintf(intercept, "%0.7lf", teta_0_intercept);
sprintf(grad, "%0.7lf", teta_1_grad);
sprintf(xrange, "%0.7lf", max_x + 1);
sprintf(yrange, "%0.7lf", max_y + 1);
concatenate(cmd_gnu_1, intercept);
concatenate(cmd_gnu_2, grad);
concatenate(cmd_gnu_3, xrange);
concatenate(cmd_gnu_3, "]");
concatenate(cmd_gnu_4, yrange);
concatenate(cmd_gnu_4, "]");
printf("grad = %s\n", grad);
printf("intercept = %s\n", intercept);
printf("xrange = %s\n", xrange);
printf("yrange = %s\n", yrange);
printf("cmd_gnu_0: %s\n", cmd_gnu_0);
printf("cmd_gnu_1: %s\n", cmd_gnu_1);
printf("cmd_gnu_2: %s\n", cmd_gnu_2);
printf("cmd_gnu_3: %s\n", cmd_gnu_3);
printf("cmd_gnu_4: %s\n", cmd_gnu_4);
printf("cmd_gnu_5: %s\n", cmd_gnu_5);
printf("cmd_gnu_6: %s\n", cmd_gnu_6);
/* print plot */
FILE *gnuplot_pipe = (FILE*)popen("gnuplot -persistent", "w");
FILE *temp = (FILE*)fopen("data.temp", "w");
/* create data.temp */
size_t i;
for (i = 0; i < size; i++)
{
fprintf(temp, "%f %f \n", vector_x[i], vector_y[i]);
}
/* display gnuplot */
for (i = 0; i < 7; i++)
{
fprintf(gnuplot_pipe, "%s \n", commands_gnuplot[i]);
}
}
int main(void)
{
printf("===========================================\n");
printf("INPUT DATA\n");
printf("===========================================\n");
user_input();
display_vector();
printf("\n");
printf("===========================================\n");
printf("COMPUTE MEAN X:Y, TETA_1 TETA_0\n");
printf("===========================================\n");
compute_mean_x_y();
compute_max_x_y();
compute_teta_1_grad();
compute_teta_0_intercept();
printf("\n");
printf("===========================================\n");
printf("COMPUTE LOSS FUNCTION\n");
printf("===========================================\n");
compute_loss_function();
printf("===========================================\n");
printf("COMPUTE R_square\n");
printf("===========================================\n");
compute_r_square(size);
printf("\n");
printf("===========================================\n");
printf("COMPUTE y^ according to x\n");
printf("===========================================\n");
compute_predict_for_x();
printf("\n");
printf("===========================================\n");
printf("DISPLAY LINEAR REGRESSION\n");
printf("===========================================\n");
display_model_line();
printf("\n");
return 0;
}
Look at Section 1 of this paper. This section expresses a 2D linear regression as a matrix multiplication exercise. As long as your data is well-behaved, this technique should permit you to develop a quick least squares fit.
Depending on the size of your data, it might be worthwhile to algebraically reduce the matrix multiplication to simple set of equations, thereby avoiding the need to write a matmult() function. (Be forewarned, this is completely impractical for more than 4 or 5 data points!)
The fastest, most efficient way to solve least squares, as far as I am aware, is to subtract (the gradient)/(the 2nd order gradient) from your parameter vector. (2nd order gradient = i.e. the diagonal of the Hessian.)
Here is the intuition:
Let's say you want to optimize least squares over a single parameter. This is equivalent to finding the vertex of a parabola. Then, for any random initial parameter, x0, the vertex of the loss function is located at x0 - f(1) / f(2). That's because adding - f(1) / f(2) to x will always zero out the derivative, f(1).
Side note: Implementing this in Tensorflow, the solution appeared at w0 - f(1) / f(2) / (number of weights), but I'm not sure if that's due to Tensorflow or if it's due to something else..

Resources