Preliminary materials

During summer school, we will discuss quite advanced topics of Bayesian Deep Learning. The school lasts only 6 days, and we do not have enough time for a detailed introduction into basic Bayesian Methods and Deep Learning. That's why we expect you to come with the knowledge of some basic concepts.

Below we present a checklist of concepts which you have to be familiar with before the summer school. We also provide links to the materials on these topics. We recommend studying the materials on concepts you are not familiar with. Otherwise, it will be hard for you to follow the school lectures.

Although there are a few intro lectures on Bayesian approach in the program, they are not enough to understand the specifics of this approach if it is your first acquaintance. But they will be helpful to understand the connections between methods if you studied the materials in advance.

Checklist of basic concepts

  1. Bayesian Methods
    • Concepts
      1. Bayesian reasoning v. s. frequentist reasoning
      2. Conjugate priors and analytical Bayesian inference
      3. EM-algorithm
      4. Variational inference, mean-field approximation, and conditional conjugacy
    • Materials
      1. First 3 weeks of Bayesian Methods for Machine Learning Course on Coursera.
      2. Or, if you prefer reading books, chapters 1, 2, 3, 9, 10 of Pattern Recognition and Machine Learning by Christopher Bishop
      3. Or, if you prefer concise lecture notes of university courses, we'd recommend this course or this course.
    • To test your knowledge, try solving Bayesian methods problems in this document.
  2. Deep Learning
  3. PyTorch
    • Concepts
      1. Tensors, autograd and implementing custom algorithms using basic operations
      2. nn.Module and how it helps to implement any computational graph
      3. Constructing and training neural networks
    • Materials
      1. Parts of an official PyTorch tutorial: Quick 60-min intro, Learning PyTorch with examples, What is torch.nn really?
      2. Or, if you prefer tutorials in Jupiter notebooks where you write missing code, we suggest solving Part Q5 (PyTorch version) of Stanford's assignment.
    • To test your knowledge, try implementing Multi-layer perceptron on the Boston dataset yourself in low-level PyTorch using nn.Module, basic mathematical operations (like matrix multiplication or random tensor initialization) and autograd.

We also recommend taking a look at a the Matrix Cookbook. It is a good collection of useful mathematical facts that will be useful at the summer school. You should understand basic concepts from this book (basic operations with matrices, matrix derivatives, properties of distributions etc).

School material

There are two types of classes at the summer school: lectures and practical sessions. For the practical sessions, please bring your laptop with python 3.7, pytorch (version 1.1) and torchvision installed. For python 3.7, we recommend installing an Anaconda distribution.

Also, please download a MNIST dataset before the school. We recommend running this script to test your installation, and the script will also download MNIST using torchvision wrapper (in the same manner as we will load data at the school).

School materials (lectures and assignments) will be available in the GitHub repository.

Tourist information

Recommended hotels

Recommended hostels

Resources for navigation and sightseeing

Other info

  • Please check the weather forecast before the school. A warm sweater may be needed.
  • Bank cards are accepted almost everywhere, including public transport
  • To get from the airport, you can use Aeroexpress, taxi (see above) or bus. When called using a mobile app, the taxi may cost not much more than Aeroexpress. The bus goes to the nearest metro station, and it is the cheapest option.
  • To get cheaper tickets for public transport, buy Troika card. Find more information about tickets here.
  • Do not forget to purchase medical insurance.
  • Mobile data plans are very cheap in Russia compared to the rest of the world. If you'd like to get yourself a local SIM card, this guide might be useful.

Venue

The School will be held at the InLiberty space at Stolyarnyy Pereulok, 3/1, Moscow near the metro station "Ulitsa 1905 goda" (Purple line)

Contact

If you have any further questions about our summer school, feel free to send us an email to info@deepbayes.ru.

You can also follow us on Twitter.