Recommended reading: The pdf Linear regression.

Slides: Download.

After this lecture:

You can start looking at Exercise Session 1, Problem 1a, 2a, 2b, 3a.

You can start looking at Computer lab 1, Section 3, 4.2, 4.3, 4.4, 4.5

Interesting read*:

P. Mattsson, D. Zachariah, P. Sotica, Analysis of the Minimum-Norm Least-Squares Estimator and Its Double-Descent Behavior, IEEE Signal Processing Magazine, 2023.

This is a paper in lecture note form that discusses and analyses how least squares behave when there are fewer samples than parameters to estimate. It only uses standard linear algebra to derive the results and also introduces how the Moore-Penrose inverse works.

It gives one relatively straightforward explanation for the so-called "double descent behavior of least squares" that has received a lot of attention in the research literature recently.

* I hope it is a bit interesting at least :)