Description

Data is becoming more widely available which opens up for game changing possibilities to teach machines to autonomously analyze, learn, and act based this data without human intervention. The development of these data-driven methods has enabled solutions to previously unsolved problems and are by many considered to be the new electricity of today’s society. The most popular and successful set of methods driving this revolution is called deep learning. Today deep learning methods outperform domain-specific techniques in a broad set of areas ranging from medicine, physics and biology to information technology, engineering science and computer science making. This makes knowledge of deep learning relevant for most scientific fields dealing with data.

The weekly lectures will be focused on theoretical aspects and the mandatory hand-in assignments on implementation of deep learning methods. The course will therefore deal less with particular applications within the field. That is where the optional project comes into the picture, where you can use the foundation provided for an application from your own scientific area.

Learning outcomes

After passing the course the student should be able to:

describe and use backpropagation together with gradient descent and stochastic gradient descent to optimize a model
implement a fully interconnected multi-layer neural network
explain under- and overfitting and what can be done to avoid them
describe and use different kinds of regularization techniques
describe and use deep convolutional networks for classification and regression
describe and use deep learning models for timeseries data
use modern environments for deep machine learning to solve practical problems

Contents

Basics of machine learning
Feed forward neural networks
Backpropagation
Stochastic gradient descent
Bias-/variance trade off, regularization
Batch normalization
Convolutional neural networks
Deep time series models

Course Structure

The course gives 5 hp (you can receive an additional 3 hp by carrying out a project).

Lectures: 9
Hand-in assignments: 3 ( +one pre-course assignment)
Helpdesks: 8
Project: Optional

Examination

The examination consists of three hand-in assignments.

Course literature

If you want a book to read along with the lecutres one good book is
[GBC] Ian Goodfellow, Yoshua Bengio and Aaron Courville Deep Learning, MIT Press, 2016.

For the first lectures, the material is also well covered by the following text book
[LWLS] Andreas Lindholm, Niklas Wahlström, Fredrik Lindsten, and Thomas B. Schön Supervised machine learning

Schedule

Lectures will take place Wednesdays at 13:15-15:00 with start March 17, 2021. The lectures will take place on zoom.

Periodicity

Every 2 years. Previous edition has been given at Uppsala University (2019). The course is given next time in spring 2023.

Course level

This is a PhD level course.

Prerequisites

Basic undergraduate courses in linear algebra, statistics, probability, optimization and programming experience in Python, MATLAB or similar.

Sign up

Apply for the course by sending en email to Niklas Wahlström. For your application to be complete you then need to submit the solutions to the following small pre-course assignment no later than March 1. We use the first come, first served principle but if we get more accepted applications than the maximum number of participants we might favor an equal distribution of participants from the different departments. Maximum number of participants is 50.

Course Projects

On September 29, 2021 course projects where presented in form of an online poster event. You find all reports and posters here.