In order to obtain the most reliable results, the number of measurements is often more than the numbers of unknown parameters, which means that the number of residual equations is more than the number of unknown numbers. The method of general solving algebraic equations cannot be used to solve these problems. The least squares method convert the residual equation into an algebraic equation with a definite solution and the unknown parameters can be solved. This algebraic equation with definite solution is called the normal equations of least squares estimation. In this paper, we build the basis of linear algebra, and propose another method for solving the least squares estimator.
THE INTRODUCTION OF THE LEAST SQUARES PRINCIPLE
The least square method is a method that estimates the regression equation using the sample data according to the least squares criterion.
Suppose Li is the i-th sample observed value and X̂ is the corresponding i-th sample best estimate. The residual between Li and X̂ is denoted by ei.
The criteria that apply the quadratic sum of the residual of all the observed values is minimized, to determine the estimates of the unknown parameter x1, x2, x3,…, xk. And the the least squares criterion is
The standard deviation of the direct measurement result is
The corresponding standard deviation estimate can be obtained according to the equation
where djj(j = 1,2,3,4,…) is the value of diagonal of (AT A)−1 obtained above.
Two methods of calculating xi and σxi
Assume observation equation is:
where aii (i = 1,2,3)—constant, lj (j = 1,2,3,4)—measurement result, which is also a constant. List the coefficient matrix A and the measurement result matrix L 
Eq.(1) shows that AT A and AT L should be known, then determine the inverse matrix C of AT A:
The least square estimator of X̂ are obtained by multiplying C by AL.
First, make a table according to the observation equation, where aii (i = 1,2,3,4) and lj (j = 1,2,3,4) are all four numbers of the corresponding columns of the constants Ai(i = 1,2,3) and Y, which are given by the four formulas above are assumed (Note: They are not a vector).
Next, make another form firstly. Ai × Aj in first row represents the four numbers, which multiply the corresponding the four numbers that are represented by Ai and Aj, respectively. For example, A1 × A1 means the number of a11 × a11、 a21 × a21、 a31 × a31、 a41 × a41. And ∑i(i = 1,2,3,…) in last row represent the sum of the first four digits of the corresponding column.
Calculation table of normal equation coefficient
|A1 × A1||A1 × A2||A1 × A3||A2 × A1||A2 × A2||A2 × A3||A3 × A1||A3 × A2||A3 × A3||A1 × Y||A2 × Y||A3 × Y|
|a11 × a11||a11 × a12|
|a21 × a21|
|a31 × a31|
|a41 × a41|
|∑ 1||∑ 2||∑ 3||∑ 4||∑ 5||∑ 6||∑ 7||∑ 8||∑ 9||∑ 10||∑ 11||∑ 12|
Then the last line of the table three per group divided into four groups and the three equations are written
Third, the least squares estimator could be calculated by solving the equations obtained in the second step.
Prove: The matrix X is grouped
According to the basic characteristics of transposed and matrix
If ∑1 = a1b1, the sum of the above table corresponds to the elements in the XT X and XT Y matrices. So the finally corresponding equation set in the third step is:
Forth, the standard deviation S of the measured value is also obtained first, and the value of djj is calculated by the equation set obtained by the above form
And the value of C1 obtained by the above formula is the value of d11. Similarly, replace the value of the right side of the equation by 0 1 0 when calculating the value of d22, and replace the value of the right side of the equation by 0 0 1
Prove: In the calculation of the matrix of the inverse matrix, the defining equation is CC−1 = E, from the above we already know
(This is a symmetric matrix, and its inverse matrix is symmetric matrix as well.) Assume its inverse matrix is , then E is a unit matrix of 3 × 3. By matrix division, whenC is multiplied by the first column of C−1, the resulting value is the first column of E,
The values of t1、 t2、 t3 are the diagonal of the inverse matrix, that is, the value of djj which we need.
As can be seen from the above two proofs, the last value obtained by the second method is exactly the same as the matrix algorithm. The first method of the least squares calculation given above is entirely based on the solution of the matrix, but the second method is more use of our more familiar algebra method. In the calculation of large amount of data, usually based on the guidance of the first method using MATLAB-  and other computer software to solve the final value, when you encounter small data, or in the answer needs, the second method if you master, it will greatly reduce the difficulty of the calculation, and not easy to miscalculate, very practical. Finally, I would like to offer this document to teachers and students puzzled by the least squares of the matrix algorithm.
CHEN Lanfeng, YANG Jingyu. [MATLAB simulation of curve fitting based onleast-squares] Journal of Shenyang Normal University (Natural Science Edition), (2014)Google Scholar
XU Yitang, [Curve Fitting Based on Least Squares Method and Its Application in MATLAB] Jiangsu University of Science and Technology, School of Electronic and Information (2012)Google Scholar
WANG Peng, “Strategy of Stimulating Studying Interest for Professional Course Teacher in Food Science and Engineering,” Journal of Anqing Teachers College (Natural Science Edition), 21(1) (2015)Google Scholar
YUE Kui, “Programming for Evaluation of Roundness Error by Least Square Mean Circle Method,” Journal of Hefei University of Technology,4, (2006)Google Scholar
LIU Zhi-ping, SHI Lin-ying, “The Principle of Least Square Algorithm and its Achievement by MATLAB,” 33–34,7(19),(2008)Google Scholar