Description

Data is becoming more widely available which opens up for game changing possibilities to teach machines to autonomously analyze, learn, and act based on this data without human intervention. The development of these data-driven methods has enabled solutions to previously unsolved problems and are by many considered to be the new electricity of today’s society. The most popular and successful set of methods driving this revolution is called deep learning. Today deep learning methods outperform domain-specific techniques in a broad set of areas ranging from medicine, physics and biology to information technology, engineering science and computer science making. This makes knowledge of deep learning relevant for most scientific fields dealing with data.

The lectures will be focused on theoretical aspects and the mandatory hand-in assignments on implementation of deep learning methods. The course will therefore deal less with particular applications within the field. That is where the optional project comes into the picture, where you can use the foundation provided for an application from your own scientific area.

Learning outcomes

After passing the course the student should be able to:

describe and use backpropagation together with gradient descent and stochastic gradient descent to optimize a model
implement a fully interconnected multi-layer neural network
explain under- and overfitting and what can be done to avoid them
describe and use different kinds of regularization techniques
describe and use deep convolutional networks for classification and regression
describe and use deep learning models for timeseries data
use modern environments for deep machine learning to solve practical problems

Contents

Basics of machine learning
Feed forward neural networks
Backpropagation
Stochastic gradient descent
Bias-/variance trade off, regularization
Batch normalization
Convolutional neural networks
Deep time series models

Course Structure

The course gives 5 hp (you can receive an additional 3 hp by carrying out a project).

Lectures: 9
Hand-in assignments: 3 ( +one pre-course assignment) (mandatory)
Peer-reviews: 3 x 2 (optional)
Problem sessions and helpdesks: ~10
Project: Optional

NAISS-UPPMAX are providing the compute resources for the course participants during the course. Get access by following the steps explained here.

Communication outside lectures

Helpdesks via zoom, Thursdays 16:15-17:00:
Discussion forum for questions in between helpdesks
Slack-space for informal discussions

You find links for zoom, slack and discussion forums here.

Examination

The examination consists of three hand-in assignments + one pre-course assignment.

Course literature

If you want a book to read along with the lecutres one good book is
[GBC] Ian Goodfellow, Yoshua Bengio and Aaron Courville Deep Learning, MIT Press, 2016.

For the first lectures, the material is also well covered by the following text book
[LWLS] Andreas Lindholm, Niklas Wahlström, Fredrik Lindsten, and Thomas B. Schön Machine Learning - A First Course for Engineers and Scientists

Schedule

The main 5hp course will be given during the second half of the spring term 2023. The lectures and helpdesks will be on campus and scheduled into three blocks to facilitate for students travelling from other universities/campuses. The dates for these three blocks will be March 22-23, April 26-27 and May 17. The optional project will be done during summer 2023 with with project proposal presentations on June 14 and final presentations early fall 2023.

Periodicity

Every 2 years. Previous edition has been given at Uppsala University in 2021 and 2019. As of 2025, the course will be given as a master course in period 3. Contact the course responsible Niklas Wahlström if you are interested following the master course as a PhD student.

Course level

This is a PhD level course.

Prerequisites

Basic undergraduate courses in linear algebra, statistics, probability, optimization and programming experience in Python, MATLAB or similar.

Time allocation

The required time to complete the course is highly depended on your previous programming experience, which usually is fairy diverse among the course participants. In the table below an estimate for time allocation for the different course component is presented.

8h pre-course assignment
30h*3 = 90h assignments
2h*3 = 6h peer-reviewing
2h*9 =18h lectures
12h reading
=134h = 3.3 weeks = 5hp.

Sign up

Sign up here no later than February 1 2023. We still have a few slots left. You can sign up until March 1, 2023, or until the max capacity of 50 students is reached.

For your application to be complete you then need to submit the solutions to the following small pre-course assignment no later than March 1 March 8, 2023. After February 1 we will register you to the course after which you can submit the pre-course assignment via Studium, but you can start working on it already now.

We use the first come, first served principle but if we get more accepted applications than the maximum number of participants we might favor an equal distribution of participants from the different departments. Maximum number of participants is 50.

Acknowledgement

The computations will be enabled by resources provided by the National Academic Infrastructure for Supercomputing in Sweden (NAISS) at Uppsala partially funded by the Swedish Research Council through grant agreement no. 2022-06725.