SimuVeda

Mathematics · Simulator 051

Regression

Tap to add data points, drag them around, and watch the best-fit line respond. See the residuals (the vertical misses), the squared errors the line minimises, and how a single outlier can swing the whole fit.

How the model works

Ordinary least squares chooses the line ŷ = a + bx that minimises the sum of squared residuals Σ(yᵢ − ŷᵢ)²:

b = Σ(x − x̄)(y − ȳ) / Σ(x − x̄)², a = ȳ − b x̄

R² is the fraction of the variation in y explained by the line (1 = perfect, 0 = none); it equals the squared correlation r². Squaring the residuals means points far from the line pull very hard. An outlier with an extreme x value (high leverage) can drag the slope a long way.

The optional squares view draws each squared residual as an actual square, so you can see what “least squares” minimises. The robust option fits with least absolute deviations instead (iteratively reweighted), which is much less sensitive to outliers.

Correlation is not causation, and a straight line can fit curved data badly. Always look at the residuals.

Guided experiments

  1. Clean linear data. Generated points around y = 2 + 0.5x. Prediction: the fitted slope is close to 0.5, R² is high, and residuals scatter randomly.
  2. One outlier with leverage. Same data plus one far-right point well below the trend. Prediction: the least-squares slope drops sharply and R² falls. The robust fit barely moves.
  3. Curved data. Points on a parabola. Prediction: a straight line fits poorly, and the residuals show a clear U-shaped pattern, the sign of a wrong model.

Challenge

Four points: (0, 1), (1, 3), (2, 5), (3, 7). What are the slope, intercept and R²? Now drag (3, 7) up to (3, 11). How does the slope change?

Reveal the explanation

The points lie exactly on y = 1 + 2x, so b = 2, a = 1 and R² = 1. Moving the last point to y = 11 adds 4 to one y value: the slope rises by 4 × (3 − 1.5)/Σ(x − x̄)² = 6/5 = 1.2, to 3.2, and R² drops below 1. Enter the points with “Clear” and tap to check.

References

  • OpenStax, Introductory Statistics, ch. 12 “Linear Regression and Correlation”.