4.3.1 Linear Least Square Regression Technique (LLS)
The least square approach is employed to determine the parameters for obtaining the ”best fitting line” for a series of observations. It aims to train parameter \(\beta_{o}\) and \(\beta_{1}\) in a linear equation as given below:
\(y=\beta_{o}+\beta_{1}x\) …… (16)
Let us consider n observations of x and y to build a mathematical model using the parameters \(\beta_{o}\)and \(\beta_{1}\), such that:
\(y_{1}=\beta_{o}+\beta_{1}x_{1}+r_{1}\)\(y_{2}=\beta_{o}+\beta_{1}x_{2}+r_{2}\)… …… (17)\(y_{n}=\beta_{o}+\beta_{1}x_{n}+r_{n}\)
where \(r_{i}\) is the residual error terms, \(\beta_{o}\) is the is the intercept of the line with the y axis, and \(\beta_{1}\) is the slope of the line.
The corresponding cost function is defined as:
\(h\left(\beta_{o},\beta_{1}\right)=\frac{1}{2}\sum_{i=1}^{n}r_{i}^{2}=\frac{1}{2}\sum_{i=1}^{n}\left(y_{i}-\beta_{o}-\beta_{1}x_{i}\right)^{2}\)…… (18)
As the cost function \(h\left(\beta_{o},\beta_{1}\right)\) is at it minimum, the partial derivatives of\(h\left(\beta_{o},\beta_{1}\right)\) with respect to\(\beta_{o}\ \)and \(\beta_{1}\) is equal to zero. This ultimately leads to the solutions of \(\hat{\beta_{o}}\) and \(\hat{\beta_{1}}\) which satisfies the least square error to give the best fitting line for a number of given data.
In this machine learning study, the least square approach is used to train ”best fitting line” for the scattered data as depicted in Figure 8 below. The purpose is to obtain the value of \(\beta_{o}\) (the intercept of the line with the y axis) and \(\beta_{1}\) (the slope of the line), as previously introduced in Eq. (16).