Introduction to Machine Learning (89511)

Staff:

Lecturer: Yossi Keshet

Teaching assistant: Yosi Shrem and Yael Segal


Course books:

Shai Shalev-Shwartz and Shai Ben-David, "Understanding Machine Learning: From Theory to Algorithms", Cambridge University Press, 2014

Ian Goodfellow and Yoshua Bengio and Aaron Courville, "Deep Learning", MIT Press, 2016.


Syllabus:

Date Lecture Recitation
27/20/2019 Introduction -
03/11/2019 Working intuitively: ERM principle, Optimal Bayes Introduction, kNN, k-means
10/11/2019 Basic concepts: PAC learning, Finite hypotheses sets ERM Principle example: the Perceptron Algorithm
17/11/2019 How to make it work better: VC, stability, and regularization Be Practicle: binary and multiclass classification, Optimization (GD, SGD), code and visualization
24/11/2019 The large margin concept: Support Vector Machine (SVM) The large margin concept: Online algorithms, linear regression
01/12/2019 How to turn linear models to handle non-linear data: SVM and kernels Evaluation metrics, Logistic regression binary
08/12/2019 Intro to Neural Nets and the maximum likelihood estimator Canceled
15/12/2019 The Backpropagation: training deep neural nets The Backpropagation: training deep neural nets
22/12/2019 Programming: PyTorch optimization methods (momentum, adagrad, adam) + tricks (do, batchnorm)
29/12/2019 Hanukkah vacation Hanukkah vacation
05/01/2020 Learning on different scales: Convolutional Neural Nets (CNN) CNN architectures (ResNet, skip-connections, ...)
12/01/2020 Working with sequential data: Recurrent Neural Nets (RNN) RNN architectures (vanila, GRU, and LSTM)
19/01/2020 - -
26/01/2020 - -


Class notes:

  1. Lecture 1 - Introduction

  2. Lecture 2 - ERM principle and Perceptron

  3. Lecture 3 - The optimal Bayes classifier and Maximum likelihood estimator

  4. Lecture 4 - PAC learnability and Support Vector Machines

  5. Lecture 5 - Byond ERM: regularization, SRM, and MDL

  6. Lecture 6 - introduction to deep learning. See Ch.4 of Deng

  7. Lecture 7 - Convolutional neural networks (CNN) - slides are from Sandford CS231n course


Tirgul notes:

A discussion group for this course is available on Piazza Q&A.

The submission should be using the Submit system