Linear Curve Fits

INTRODUCTION

After taking measurements in the presence of noise or other inaccuracies, we often want to find a function which best matches or predicts that data.  In mathematics, the exercise of adjusting parameters in some predicting function for this purpose is called “curve-fitting” or “regression.”

If we draw a straight line through the measured data points, that function is obviously “linear.”  But in a more general sense regression is said to be “linear” if multiplying the parameters by some constant, “c”, increases the total value of the function by that same constant factor. For a single variable “x” and one constant, this means In all such cases there is a unique solution for the parameter set.    That is to say for an independent variable “y” that is caused by or “determined” by the dependent variable “x”, we have “n” simultaneous measurements  of both variables  as We then want to determine the unique set of “m+1” parameters that when used in an expansion of a family of functions  will minimize the total squared error between the measurements and functional predictions as The functions are typically a set of orthogonal functions but could simply be a polynomial power series where or LEAST SQUARES CURVE FIT

We start by taking the derivatives of the squared error with respect to the each one of the “m+1” parameters as follows or by rearranging terms, we have These “m+1” equations can then be more concisely expressed in a matrix format as follows which allows for a direct calculation of the parameters using perhaps the Gauss-Jordan method.

LINEAR LEAST-SQUARES FIT

For a linear polynomial or straight line with m=1, then and the matrix is This can be directly solved as   For a quadratic polynomial with m=2, then and the matrix is But if we rewrite the matrix as then the exact solution is    LEGENDRE LEAST-SQUARES FIT

The first few Legendre polynomials are     And the Legendre polynomials satisfy the recurrence relation or more directly 