Total suggested work for these two weeks

Week 2: 4 hours lectures, 2 hours computer lab (linear regression), 2 hours of reading from course book, 2 hours paper and pencil self-study. 1 hour form mini-project group. Total 11 hours.

Week3: 2 hours lectures, 2 hours paper and pencil, 2 hours computer lab self-study, 2 hours computer lab, 2 hours start working on mini-project. Total 10 hours.

Before you start...

We prepared a self-assessment quiz with true-false statements from previous exams about important concepts of week 1. The quiz does not count towards the final grades and can be repeated multiple times. Note, however, that it does not cover everything that you should know about the topics of week 1.

2.1 Mini-project

You need to form a group for the mini-project. This is an examined component of the course and it is important we get stated early.

2.2 What is Classification?

Here is the Udacity video explaining difference between classification and regression. It explains the basic idea.

However, this video also illustrates (unknowingly it appears) one serious problem in current day machine learning: stereotyping and bias. Classifying gender based on hair length is not a valid or useful application of machine learning. We will come back to this point later in the course.

2.3 Reading

This week you should set aside two hours for reading the The Supervised Machine Learning book. At the start of this week read section 2.1 first introducing and then read section 3.1. This covers what we looked at last week.

During the week, in parallel to the lectures, read sections 3.2 and 3.3 on logistic regression and regularisation, as well as sections 2.1 on k-NN and 11.1 on discriminant analysis.

2.4 Logistic Regression

Lecture will be on Wednesday 27th of January 08:15-10:00. The video from last year available here:

Updated lecture notes from this year available here (and here with handwritten additions). This years video is here.

2.5 k-Nearest Neighbors and Discriminant Analysis

Lecture will be on 28th of January 10:15-12:00. The videos from last year available here:

k-Nearest Neighbours

Discriminant analysis

Lecture notes from this year are available here.

The video from this year is here:

2.6 Paper and Pencil Exercises

Start this work on 28th of January. Spend about two hours attempting the questions before the tutorial on Friday.

The exercise sheet is available here.

Recommended problems: 3.1, 3.2, 3.3, 3.5
Additional problems: 3.4, 3.6, 3.7

This will be covered at the tutorial on Tuesday 2nd of February 13:15-15:00. Also feel free to ask questions in the discussion section. The drop-in session directly afterwards (15:15-17:00) is a chance to ask questions.

2.7 Computer Lab

Start this work on January 29th. Spend about two hours attempting the questions before the computer lab on Thursday 4th of February.

The computer labs are available here:

Recommended problems: 4.1, 4.2, 4.3, 4.4
Additional problems: 4.5

Suggested solutions are available here.

These will be covered at the tutorial on Thursday 4th of February 08:15-10:00. Please attempt the questions before the tutorial. Also feel free to ask questions in the discussion section. The drop-in session directly afterwards (10:15-12:00) is a chance to ask further questions.

2.8 I love this and I want to do more

Geometry of logistic regression

This video gives a more in-depth mathematical/geometric interpretation of logistic regression. This is good up until the end where, I get confused what he is drawing.

Here he also uses males and females in the example, but this is not problematic as an application simply because men and women do have different health issues due to differences in biology.

Deeper understanding

Section 3.A in the SML book covers the derivation of the normal equations in more depth. The proof is done both from a minimisation and a linear algebra perspective. Section 5.2 goes in to more depth on parameter estimation.

Applications in football

I have made a series of videos about fitting an expected goals model in football using logistic regression. These are from the course 'Mathematical Modelling of Football'. Here I look at the data.

Here I fit the model using logistic regression.

See this page for more details. I cover the three main regression problems: linear, logistic and Poisson. The latter is not used in this course. You will also notice I use Statsmodels instead of Scikitlearn. Statsmodels is powerful for a maximum likelihood based approach.