Learning, Classification, and Compression

Offered in:

Data Science Master: Wählbare Kernfächer
Doktorat Departement Informationstechnologie und Elektrotechnik: Lehrangebot Doktorat und Postdoktorat
Elektrotechnik und Informationstechnologie Master: Vertiefungsfächer
Elektrotechnik und Informationstechnologie Master: Empfohlene Fächer
Mathematik Bachelor: Auswahl: Weitere Gebiete
Mathematik Master: Auswahl: Weitere Gebiete
Physik Master: Allgemeine Wahlfächer
Statistik Master: Statistische und mathematische Fächer
Statistik Master: Fachbezogene Wahlfächer

Basic Information:

Lecture:	Thursday, 11:15-13:00, HG E 33.5. The first lecture takes place on Thursday 29 Feb. 2024, 12:15-14:00, HG E 33.5.
Discussion session:	Thursday, 13:15-14:00, HG E 33.5. The first discussion session takes place on Thursday 29 Feb. 2024, 11:15-12:00.
Office hours:	Friday, 10:15-11:00 via Zoom. The first office hour takes place on Friday 01 Mar. 2024, 10:15-11:00.
Zoom Links:	The Zoom link for the office hours can be found at this page (access credentials are the same as for the lecture/exercise notes).
Instructor:	Dr. Erwin Riegler
Teaching assistant:	Rodrigo Casado Noguerales
Lecture notes:	Detailed lecture notes will be made available as we go along.
Prerequisites:	This course is aimed at students with a solid background in measure theory and linear algebra and basic knowledge in functional analysis.
Credits:	4 ECTS credits.
Course structure:	The class will be taught in English. There will be an oral exam of duration 30 minutes.

Course Information:

The focus of the course is aligned to a theoretical approach of learning theory and classification and an introduction to lossy and lossless compression for general sets and measures. We will mainly focus on a probabilistic approach, where an underlying distribution must be learned/compressed. The concepts acquired in the course are of broad and general interest in data sciences.

After attending this lecture and participating in the exercise sessions, students will have acquired a working knowledge of learning theory, classification, and compression.

News

We will post important announcements, links, and other information here in the course of the semester, so please check back often!

There is no lecture and no discussion session on Feb. 22. We will find a replacement date for the first lecture.
Problems 8 and 9 in the discussion session got swapped.
From this year on, the exam will be oral.
The time slots for lecture and discussion session got swapped.

Content of the Course:

Learning Theory:

Framework of Learning
Hypothesis Spaces and Target Functions
Reproducing Kernel Hilbert Spaces
Bias-Variance Tradeoff
Estimation of Sample and Approximation Error

Classification:

Binary Classifier
Support Vector Machines (separable case)
Support Vector Machines (nonseparable case)
Kernel Trick

Lossy and Lossless Compression:

Basics of Compression
Compressed Sensing for General Sets and Measures
Quantization and Rate Distortion Theory for General Sets and Measures

Prerequisites

This course is aimed at students with a solid background in measure theory and linear algebra and basic knowledge in functional analysis.

Lecture Notes and problems+solutions:

Problem sets and solutions

There will be several problem sets for this course, which will help you better understand the lectures and prepare you for the exam (See download link Problems + Solutions above). The following problem will be discussed in the discussion session:

Discussion Session:	Problem:
29.02.2024	Problem 1
07.03.2024	Problem 2
14.03.2024	Problem 3
21.03.2024	Problem 4
28.03.2024	Problem 5
11.04.2024	Problem 6
18.04.2024	Problem 7
25.04.2024	Problem 9
02.05.2024	Problem 8
16.05.2024	Problem 10
23.05.2024	Problem 11
30.05.2025	Problem 12

Previous years' exams and solutions

Summer Exam 2021:	Problems	Solutions	Handout
Summer Exam 2022:	Problems	Solutions	Handout
Summer Exam 2023:	Problems	Solutions	Handout