Linear Curve Fits

INTRODUCTION

After taking measurements in the presence of noise or other inaccuracies, we often want to find a function which best matches or predicts that data.  In mathematics, the exercise of adjusting parameters in some predicting function for this purpose is called “curve-fitting” or “regression.”

If we draw a straight line through the measured data points, that function is obviously “linear.”  But in a more general sense regression is said to be “linear” if multiplying the parameters by some constant, “c”, increases the total value of the function by that same constant factor. For a single variable “x” and one constant, this means

In all such cases there is a unique solution for the parameter set.    That is to say for an independent variable “y” that is caused by or “determined” by the dependent variable “x”, we have “n” simultaneous measurements  of both variables  as

We then want to determine the unique set of “m+1” parameters

that when used in an expansion of a family of functions

will minimize the total squared error between the measurements and functional predictions as

The functions  are typically a set of orthogonal functions but could simply be a polynomial power series where

or

LEAST SQUARES CURVE FIT

We start by taking the derivatives of the squared error with respect to the each one of the “m+1” parameters as follows

or by rearranging terms, we have

These “m+1” equations can then be more concisely expressed in a matrix format as follows

which allows for a direct calculation of the parameters using perhaps the Gauss-Jordan method.

LINEAR LEAST-SQUARES FIT

For a linear polynomial or straight line with m=1, then

and the matrix is

This can be directly solved as

For a quadratic polynomial with m=2, then

and the matrix is

But if we rewrite the matrix as

then the exact solution is

LEGENDRE LEAST-SQUARES FIT

The first few Legendre polynomials are

And the Legendre polynomials satisfy the recurrence relation

or more directly