Least Squares Lines
Source inspiration: (Mathew 2000-2019).
Description
Least squares line fitting chooses parameters that minimize the sum of squared residuals between measured data and a model. The standard linear model is \(y = b + a x\), where residuals are vertical distances from points to the fitted line.
This module also highlights two important variations:
- Regressing \(x\) on \(y\) instead of \(y\) on \(x\) produces a different line after rearrangement, even for the same data.
- Constrained models such as \(y = a x\) and \(y = a x^{3/2}\) remain linear in the unknown coefficient \(a\), so least squares still applies directly.
Animations
Each animation below shows how the fitted model changes as data points are added. Residuals and fitted parameters update frame-by-frame so you can see what the least-squares criterion is doing geometrically.
Case 1 - Standard Least-Squares Line, \(y = b + a x\)
Behavior: The unconstrained fit minimizes vertical residuals to a straight line. As points are added, the slope and intercept stabilize near \(a \approx -1.6071\) and \(b \approx 8.6429\).
Case 2 - Two Regression Lines and Their Intersection
Behavior: Fitting \(y\) as a function of \(x\) and fitting \(x\) as a function of \(y\) produce two distinct lines in the \(x\)-\(y\) plane. Their intersection coincides with the centroid of the data, here approximately \((\bar{x}, \bar{y}) = (2.5, 4.625)\).
Case 3 - Constrained Fit Through the Origin, \(y = a x\)
Behavior: Constraining the intercept to zero changes the optimization geometry and produces a single-parameter fit. The coefficient settles near \(a \approx 1.5392\) for the legacy dataset.
Case 4 - Kepler-Style Power Fit, \(T = a r^{3/2}\)
Behavior: With exponent fixed at \(3/2\), the power law is linear in \(a\) after choosing basis function \(r^{3/2}\). Using Mercury, Venus, Earth, and Mars data gives \(a \approx 0.19977\).
Derivation Notes (Planned)
Short derivations will be added to explain the core equations and assumptions.
Worked Example (Planned)
A compact numerical example with intermediate steps will be included.
Implementation Notes (Planned)
Implementation details, numerical stability notes, and practical pitfalls will be added.