Advanced Topics in Deep Learning (896542)


Staff:

Lecturer: Yossi Keshet <jkeshet@cs.biu.ac.il>

Teaching assistant: Roni Cheriyak <advmlcourses+adv@gmail.com>

Zoom link to the live meeting is https://us02web.zoom.us/j/86973527069?pwd=UkFMVkxzRFR2cUR6R08zNjRBMWJMUT09


Lecture notes

  1. Deep learning theory 1: What is machine learning theory? What are generalization bounds?
    Definitions and introduction.
    Recording of the lecture held on 8/3/2021.

  2. Deep learning theory 2: Re-thinking generalization; Robustness and generalization; Stability and generalization
    The lecture is based on the following resources:
    - Zhang, Bengio, Hardt, Recht and Vinyals, Understanding deep learning requires rethinking generalization, 2017
    - Jakubovitz, Giryes, and Rodrigues, Generalization Error in Deep Learning, 2018
    - Xu and Mannor, Robustness and Generalization, 2012
    - The blog of Mostafa Samir (three parts), 2018
    Recording of the lecture held on 15/3/2021.

  3. Deep learning theory 3: PAC-Bayes theory and generalization
    The lecture is based on the following resources:
    - Belkin, Hsu, Ma, and Mandal, Reconciling modern machine-learning practice and the classical bias–variance trade-off, 2019
    - Jiang, Neyshabur, Mobahi, Krishnan, and Bengio, Fantastic Generalization Measures and Where to Find Them, 2019
    Recording of the lecture held on 6/4/2021.