4.3.1 Linear Least Square Regression Technique (LLS)
The least square approach is employed to determine the parameters for
obtaining the ”best fitting line” for a series of observations. It aims
to train parameter \(\beta_{o}\) and \(\beta_{1}\) in a linear equation
as given below:
\(y=\beta_{o}+\beta_{1}x\) …… (16)
Let us consider n observations of x and y to build a mathematical model
using the parameters \(\beta_{o}\)and \(\beta_{1}\), such that:
\(y_{1}=\beta_{o}+\beta_{1}x_{1}+r_{1}\)\(y_{2}=\beta_{o}+\beta_{1}x_{2}+r_{2}\)… …… (17)\(y_{n}=\beta_{o}+\beta_{1}x_{n}+r_{n}\)
where \(r_{i}\) is the residual error terms, \(\beta_{o}\) is the is the
intercept of the line with the y axis, and \(\beta_{1}\) is the slope of
the line.
The corresponding cost function is defined as:
\(h\left(\beta_{o},\beta_{1}\right)=\frac{1}{2}\sum_{i=1}^{n}r_{i}^{2}=\frac{1}{2}\sum_{i=1}^{n}\left(y_{i}-\beta_{o}-\beta_{1}x_{i}\right)^{2}\)…… (18)
As the cost function \(h\left(\beta_{o},\beta_{1}\right)\) is at it
minimum, the partial derivatives of\(h\left(\beta_{o},\beta_{1}\right)\) with respect to\(\beta_{o}\ \)and \(\beta_{1}\) is equal to zero. This ultimately leads
to the solutions of \(\hat{\beta_{o}}\) and \(\hat{\beta_{1}}\) which
satisfies the least square error to give the best fitting line for a
number of given data.
In this machine learning study, the least square approach is used to
train ”best fitting line” for the scattered data as depicted in Figure 8
below. The purpose is to obtain the value of \(\beta_{o}\) (the
intercept of the line with the y axis) and \(\beta_{1}\) (the slope of
the line), as previously introduced in Eq. (16).