Deep learning

Deep Learning (DL) PhD course (5+3hp)

Description

Data is becoming more widely available which opens up for game changing possibilities to teach machines to autonomously analyze, learn, and act based this data without human intervention. The development of these data-driven methods has enabled solutions to previously unsolved problems and are by many considered to be the new electricity of today’s society. The most popular and successful set of methods driving this revolution is called deep learning. Today deep learning methods outperform domain-specific techniques in a broad set of areas ranging from medicine, physics and biology to information technology, engineering science and computer science making. This makes knowledge of deep learning relevant for most scientific fields dealing with data.

The weekly lectures will be focused on theoretical aspects and the mandatory hand-in assignments on implementation of deep learning methods. The course will therefore deal less with particular applications within the field. That is where the optional project comes into the picture, where you can use the foundation provided for an application from your own scientific area.

Learning outcomes

After passing the course the student should be able to:

  • describe and use backpropagation together with gradient descent and stochastic gradient descent to optimize a model
  • implement a fully interconnected multi-layer neural network
  • explain under- and overfitting and what can be done to avoid them
  • describe and use different kinds of regularization techniques
  • describe and use deep convolutional networks for classification and regression
  • describe and use deep learning models for timeseries data
  • use modern environments for deep machine learning to solve practical problems

Contents

  • Basics of machine learning
  • Feed forward neural networks
  • Backpropagation
  • Stochastic gradient descent
  • Bias-/variance trade off, regularization
  • Batch normalization
  • Convolutional neural networks
  • Deep time series models

Course Structure

The course gives 5 hp (you can receive an additional 3 hp by carrying out a project).

Examination

The examination consists of three hand-in assignments.

Course literature

If you want a book to read along with the lecutres one good book is
[GBC] Ian Goodfellow, Yoshua Bengio and Aaron Courville Deep Learning, MIT Press, 2016.

For the first lectures, the material is also well covered by the following text book
[LWLS] Andreas Lindholm, Niklas Wahlström, Fredrik Lindsten, and Thomas B. Schön Supervised machine learning

Schedule

Lectures will take place Wednesdays at 13:15-15:00 with start March 17, 2021. The lectures will take place on zoom.

Periodicity

Every 2 years. Previous edition Links to an external site. has been given at Uppsala University (2019). The course is given next time in spring 2023.

Course level

This is a PhD level course.

Prerequisites

Basic undergraduate courses in linear algebra, statistics, probability, optimization and programming experience in Python, MATLAB or similar.

Sign up

Apply for the course by sending en email to Niklas Wahlström. For your application to be complete you then need to submit the solutions to the following small pre-course assignment no later than March 1. We use the first come, first served principle but if we get more accepted applications than the maximum number of participants we might favor an equal distribution of participants from the different departments. Maximum number of participants is 50.

Course Projects

On September 29, 2021 course projects where presented in form of an online poster event. You find all reports and posters here.