Residuals and what least squares minimises

Residuals and least squares

For a data point, the residual is the actual response value minus the value predicted by the fitted line at the same explanatory value. Least squares chooses the line that minimises the sum of squared residuals

1. Predict at the same explanatory value Substitute the point's explanatory value into the fitted line to find its predicted response \(\displaystyle \hat{y}=a+b\left(x_i\right)\)
2. Subtract predicted from actual Calculate the residual as the actual response value minus the predicted response value \(\displaystyle e_i=y_i-\hat{y}_i\)
3. Keep the sign A positive residual is above the fitted line and a negative residual is below it \(\displaystyle e_i>0\Rightarrow\text{above the line},\quad e_i<0\Rightarrow\text{below the line}\)
4. Square before adding Least squares minimises the sum of squared residuals so positive and negative residuals cannot cancel \(\displaystyle \text{least squares: minimise }\sum e_i^{2}\)

Worked example

A fitted line predicts processing time in minutes from the number of parcels handled. The line and one case are shown below. Calculate its residual and state what least squares minimises

1. \(\displaystyle \hat{y}=8+0.4\times15=8+6=14\)Substitute the explanatory value 15 into the fitted line to find the predicted time
2. \(\displaystyle e=y-\hat{y}=12-14=-2\)Subtract predicted time from actual time, giving a negative residual because the point is below the line
3. \(\displaystyle e^{2}=\left(-2\right)^{2}=4\)Squaring this residual makes its contribution to the least-squares total positive
4. \(\displaystyle \text{least squares minimises }\sum e_i^{2}\)The fitted line is chosen to make the total of all squared vertical residuals as small as possible

Common mistakes

Difficulty levels

Questions about this topic

In this chapter