Total suggested work for this week

4 hours lectures, 3 hours computer lab self-study, 2 hours of reading from course book, 2 hours paper and pencil self-study, 2 hours pen and paper tutorial session, 2 hours computer lab session. Total 15 hours.

Before you start...

We prepared a self-assessment quiz with true-false statements from previous exams about important concepts of week 1. The quiz does not count towards the final grades and can be repeated multiple times. Note, however, that it does not cover everything that you should know about the topics of week 1.

2.1 Mini-project

Go here for details of the mini-project. You need to form a group. Please do this by 10th of November. This is an examined component of the course and it is important we get stated early.

2.2 What is Classification?

Here is the Udacity video explaining difference between classification and regression. It explains the basic idea.

However, this video also illustrates (unknowingly it appears) one of the biggest problems in current day machine learning: stereotyping and bias. Classifying gender based on hair length is not a valid or useful application of machine learning. We will come back to this point in week 8.

2.3 Reading

This week you should set aside two hours for reading the The Supervised Machine Learning book. At the start of this week read section 2.1 first introducing and then read section 3.1. This covers what we looked at last week.

During the week, in parallel to the lectures, read sections 3.2 and 3.3 on logistic regression and regularisation, as well as sections 2.1 on k-NN and 11.1 on discriminant analysis.

2.4 Logistic Regression

Lecture will be on Tuesday 3th of November 13:15-15:00. The video will be posted here afterwards.

Lecture notes available here.

2.5 k-Nearest Neighbors and Discriminant Analysis

Lecture will be on Thursday 5th of November 13:15-15:00. Please note this is an extra lecture (to replace the one disturbed by the fire alarm) and may clash with your other lessons. I was going to just film it and put it up, but realised that it is better to do these live!

k-Nearest Neighbours

Discriminant analysis

I updated the lecture notes after the lecture because I didn't explain how to estimate the covariance (for one class) very well. New slides with covariance estimate slide here.

2.6 Paper and Pencil Exercises

Start this work on Thursday 5th November. Spend about two hours attempting the questions before the tutorial on Friday.

The exercise sheet is available here.

Recommended problems: 3.1, 3.2, 3.3, 3.5
Additional problems: 3.4, 3.6, 3.7

This will be covered at the tutorial on Friday 6th of November 10:15-12:00. Also feel free to ask questions in the discussion section.

Padlet: https://padlet.com/davidwidmann1/q05r0dswryc4vzg0

2.7 Computer Lab

Start this work on Thursday 5th November. Spend about two hours attempting the questions before the computer lab on Monday 9th of November.

The computer labs are available here:

Python notebook
Google Colab

Recommended problems: 4.1, 4.2, 4.3, 4.4
Additional problems: 4.5

Suggested solutions are available here.

These will be covered at the tutorial on Monday 9th November 10:15-12:00. Please attempt the questions before the tutorial. Also feel free to ask questions in the discussion section.

2.8 I love this and I want to do more

Geometry of logistic regression

This video gives a more in-depth mathematical/geometric interpretation of logistic regression. This is good up until the end where, I get confused what he is drawing.

Here he also uses males and females in the example, but this is not problematic as an application simply because men and women do have different health issues due to differences in biology.

Deeper understanding

Section 3.A in the SML book covers the derivation of the normal equations in more depth. The proof is done both from a minimisation and a linear algebra perspective. Section 5.2 goes in to more depth on parameter estimation.

Applications in football

I have made a series of videos about fitting an expected goals model in football using logistic regression. These are from the course 'Mathematical Modelling of Football'. Here I look at the data.

Here I fit the model using logistic regression.

See this page for more details. I cover the three main regression problems: linear, logistic and Poisson. The latter is not used in this course. You will also notice I use Statsmodels instead of Scikitlearn. Statsmodels is powerful for a maximum likelihood based approach.