Mathematics of Information
Offered in:
 Data Science Master: Information and Learning
 Computer Science Master: Elective Courses (Wahlfächer)
 Electrical Engineering and Information Technology Master: Core subjects (Kernfächer), Specialization courses (Vertiefungsfächer), Advanced core courses
 Mathematics Master: Selection: Further Realms (Auswahl: Weitere Gebiete)
 Physics Master: General Electives (Allgemeine Wahlfächer)
 Quantum Engineering Master: Electives (Wahlfächer)
 Computational Science and Engineering Master: Electives (Wahlfächer)
 Statistics Master: Statistical and Mathematical Courses (Statistische und mathematische Fächer)
 Cyber Security Master: Elective Courses (Wahlfächer)
 Doctoral and PostDoctoral Studies: Department of Information Technology and Electrical Engineering
Basic information:
Lecture:  Thursday, 9:1512:00, in room ML F 36. The first lecture takes place on Thursday, February 23, 2023. 
Exercise session:  Monday, 14:1516:00, in room ML E 12. 
Instructor:  Prof. Dr. Helmut Bölcskei 
Teaching assistants:  Clemens Hutter, Konstantin Häberle 
Office hours:  Monday, 16:1517:15 in ETF E 118. Please contact the TAs if you are planning to attend. 
Lecture notes:  Detailed lecture notes can be found below in the section "Lecture notes and prerequisite material". Exercise problem sets along with detailed solutions will be made available as we go along, see the section "Exercise sessions". The access information for the notes will be sent by email on the first day of the spring semester to all students registered for the class. 
Credits:  8 ECTS credits 
Lecture recordings:
The lectures will not be recorded. However, recordings from previous years are available below.Course structure
 The class will be taught in English. There will be a written exam in English of duration 180 minutes.
News
We will post important announcements, links, and other information here during the semester, so please check back often!
 There will be no exercise session in the first week of the semester. The class will start with the lecture on February 23, 2023 and the first exercise session will be held on February 27, 2023.
 The handwritten notes from the lecture on March 09, 2023, can be found here.
 The lectures on March 16, 2023, and March 30, 2023, will take place via zoom. Please join the lectures by clicking on this zoom link (Code: 570675).
 Until and including Monday, April 3, 2023, the exercise session takes place in ML D 28.
 The handwritten notes from the lecture on March 16, 2023, can be found here.
 The handwritten notes from the lecture on March 30, 2023, can be found here.
 The lecture on April 20, 2023, takes place from 8.15 to 10.30 in room ML F 36.
 The will be no lecture on April 27, 2023. Please watch the following recordings from 2021 instead: May 20, 2021, from 01:26:00 and May 27, 2021, from 00:25:00 to 01:41:00.

There will be a lecture on Monday, May 22, 2023, 14:1516:00, in room ML E 12 instead of the exercise session. There will be an exercise session and question hour on Thursday, June 01, 2023, 9:1512:00, in room ML F 36 instead of the lecture.
The dates for the remainder of the semester are: Monday, May 15 : Exercise
 Monday, May 22 : Exercise
 Thursday, May 25 : Lecture
 Thursday, June 1: Lecture
 A discussion on different notions of convergence of random variables can be found in (Grimmet and Stirzaker, 2001, pp. 308317).
Course information
The class focuses on mathematical aspects of information science and learning theory.
 Mathematics of Information:
 Signal representations: Frame theory, wavelets, Gabor expansions, sampling theorems, density theorems
 Sparsity and compressed sensing: Sparse linear models, uncertainty relations in sparse signal recovery, superresolution, spectrumblind sampling, subspace algorithms (ESPRIT), estimation in the highdimensional noisy case, Lasso
 Dimensionality reduction: Random projections, the JohnsonLindenstrauss Lemma
 Mathematics of Learning:
 Approximation theory: Linear and nonlinear approximation, best Mterm approximation, greedy algorithms, fundamental limits on compressibility of signal classes, KolmogorovTikhomirov epsilonentropy of signal classes, optimal compression of signal classes
 Uniform laws of large numbers: Rademacher complexity, VapnikChervonenkis dimension, classes with polynomial discrimination
H. Bölcskei and A. Bandeira
Prerequisites
This course is aimed at students with a background in linear algebra, probability, and basic functional analysis. In particular, familiarity with Hilbert spaces on the level of the "Hilbert spaces" chapter posted below (excluding the appendices) is expected.
Lecture notes and prerequisite material
 Lecture notes
 Prerequisite material
Hilbert spaces  Handouts
Comprehensive summary of linear algebra
Comprehensive summary of functional analysis
Notes on compact sets
Recordings from 2022
Date  Recording Lectures  Handwritten Notes  Recording Exercises 

03.03.2022  Lecture 1  iPad notes  Exercise 1 
10.03.2022  Lecture 2  iPad notes  Exercise 2 
17.03.2022  Lecture 3  iPad notes  Exercise 3 
24.03.2022  Lecture 4  iPad notes V2  Exercise 4 
31.03.2022  Lecture 5  iPad notes  Exercise 5 
07.04.2022  Lecture 6  iPad notes  Exercise 6 
14.04.2022  Lecture 7  iPad notes  Exercise 7 
28.04.2022  Lecture 8  iPad notes  no exercise 
05.05.2022  Part 1, from 24:41 Part 2, until 44:30 
iPad notes  Exercise 9 
12.05.2022  Lecture 10  iPad notes  Exercise 10 
19.05.2022  Lecture 11  iPad notes  Exercise 11 
02.06.2022  Lecture 12  iPad notes  Exercise 12 
Recordings from 2021
Homework assignments
There will be exercise session notes posted every Thursday after the lecture. The TAs will inform you by email which of the problems in these notes will be solved in the exercise session the following Monday. The expectation is that you prepare these problems before the exercise session.
Exercise sessions
27.02.2023  Set 1  Solutions 1  Complements on the Fourier Transform 
06.03.2023  Set 2  Solutions 2  
13.03.2023  Set 3  Solutions 3  Complements on Wavelets 
20.03.2023  Set 4  Solutions 4  
27.03.2023  Set 5  Solutions 5  
03.04.2023  Set 6  Solutions 6  
24.04.2023  Set 7  Solutions 7  
08.05.2023  Set 8  Solutions 8  
15.05.2023  Set 9  Solutions 9  Notes on compact sets 
22.05.2023  Set 10  Solutions 10 
Previous years' exams and solutions
Year  Exam  Handout  Solutions 

Summer 2018  Exam 2018    Solutions 2018 
Summer 2019  Exam 2019  Handout 2019  Solutions 2019 
Summer 2020  Exam 2020  Handout 2020  Solutions 2020 
Winter 2020/2021  Exam 2020/2021  Handout 2020/2021  Solutions 2020/2021 
Summer 2021  Exam 2021  Handout 2021  Solutions 2021 
Winter 2021/2022  Exam 2021/2022  Handout 2021/2022  Solutions 2021/2022 
Summer 2022  Exam 2022  Handout 2022  Solutions 2022 
Material for the exam
All the material that has been covered during the lectures and the exercise sessions will be relevant for the exam.
You are allowed to bring a summary of 10 handwritten or printed A4 pages (or 5 A4 pages on both sides). Electronic devices (laptops, calculators, cellphones, etc...) are not allowed.
Recommended reading
If you want to go into more depth or if you need additional background material, please check out these books:
 S. Mallat, "A wavelet tour of signal processing: The sparse way", 3rd ed., Elsevier, 2009
 M. Vetterli and J. Kovačević, "Wavelets and subband coding", Prentice Hall, 1995
 I. Daubechies, "Ten lectures on wavelets", SIAM, 1992
 O. Christensen, "An introduction to frames and Riesz bases", Birkhäuser, 2003
 K. Gröchenig, "Foundations of timefrequency analysis", Springer, 2001
 M. Elad, "Sparse and redundant representations — From theory to applications in signal and image processing", Springer, 2010
 M. Vetterli, J. Kovačević, and V. K. Goyal, "Foundations of signal processing", 3rd ed., Cambridge University Press, 2014
 S. Foucart and H. Rauhut, "A mathematical introduction to compressive sensing", Springer, 2013
 M. J. Wainwright, "Highdimensional statistics: A nonasymptotic viewpoint", Vol. 48, Cambridge University Press, 2019
 R. Vershynin, "Highdimensional probability: An introduction with applications in data science", Vol. 47, Cambridge University Press, 2018
 G. Grimmett and D. Stirzaker, "Probability and random processes", 3rd ed., Oxford University Press, 2001