Mathematics of Information

Prof. Dr. Helmut Bölcskei

Offered in:

 

Lecture:Thursday, 9:15-12:00, ETZ E6. The first two lectures take place on Monday 24 Feb 2020, 13:15-15:00, ML E 12, and on Thursday 27 Feb 2020, 9:15-12:00, ETZ E6.
Discussion session:   Monday, 13:15-15:00, ML E 12. The first discussion session takes place on Monday 2 Mar 2020, 13:15-15:00.
Instructor: Prof. Dr. Helmut Bölcskei
Teaching assistants: Verner Vlačić, Thomas Allard
Office hours: Monday, 15:15-16:15, ETF E 117 and ETF E 118
Lecture notes:Detailed lecture and exercise notes and problem sets with documented solutions will be made available as we go along.
Credits: 8 ECTS credits


Course structure


News

We will post important announcements, links, and other information here in the course of the semester, so please check back often!

Updates related to COVID-19

Both the lecture and the exercise session will be recorded from week to week and the recordings will be made available at the latest the day after the exercise (i.e. Tuesday) or lecture (i.e. Friday) by noon under the following link.

As the lecture will not be live-streamed, Prof. Dr. Helmut Bölcskei will offer a Skype or Zoom office hour every Monday 16:15-17:15. If you want to talk to him, please send an email to hboelcskei@ethz.ch before so he can communicate his Skype or Zoom id to you.

Thomas Allard and Verner Vlačić will hold their office hours via skype or zoom as well and at the regular time, i.e., Monday 15:15-16:15. If you want to talk to them, please email them so they can send you their Skype or Zoom accounts.

The compulsory research project is on. Information on how to sign up will follow.

Homeworks will be posted as planned and you may scan your solutions and email them to Verner Vlačić and Thomas Allard.



Course Info

The class focuses on mathematical aspects of information science and learning theory.

Mathematics of Information:

Signal representations: Frame theory, wavelets, Gabor expansions, sampling theorems, density theorems

Sparsity and compressed sensing: Sparse linear models, uncertainty relations in sparse signal recovery, matching pursuits, super-resolution, spectrum-blind sampling, subspace algorithms (MUSIC, ESPRIT, matrix pencil), estimation in the high-dimensional noisy case, LASSO

Dimensionality reduction: Random projections, the Johnson-Lindenstrauss Lemma

Mathematics of Learning:

Approximation theory: Nonlinear approximation theory, fundamental limits on compressibility of signal classes, Kolmogorov-Tikhomirov epsilon-entropy of signal classes, optimal compression of signal classes, recovery from incomplete data, information-based complexity, curse of dimensionality

Uniform laws of large numbers: Rademacher complexity, Vapnik-Chervonenkis dimension, classes with polynomial discrimination, blessings of dimensionality


Material to study for the exam

Here is the material that you need to prepare for the exam:



Prerequisites

This course is aimed at students with a background in linear algebra, probability, and basic functional analysis. In particular, familiarity with Hilbert spaces is expected on the level of the "Hilbert spaces" chapter posted below (excluding the appendices).


Lecture and exercise notes

Here we will post lecture and discussion session notes in due course.



Lecture and exercise video recordings

Here we will post lecture and discussion session videos. Both the lecture and the exercise session will be recorded from week to week and the recordings will be made available at the latest the day after the exercise (i.e. Tuesday) or lecture (i.e. Friday) by noon.



Homework Assignments

There will be 6 homework assignments. You can hand in your solutions and get feedback from us, but it is not mandatory to turn in solutions. Complete solutions to the homework assignments will be posted on the course web page.


Homework Problem Sets

Problems Solutions
Homework 1 Solutions to Homework 1
Homework 2 Solutions to Homework 2
Homework 3 Solutions to Homework 3
Homework 4 Solutions to Homework 4
Homework 5 Solutions to Homework 5
Homework 6 Solutions to Homework 6


Handouts

Previous years' exams and solutions



Recommended reading
If you want to go into more depth or if you need additional background material, please check out these books: