CS 6362 - Advanced Machine Learning

This is the course webpage for Advanced Machine Learning.

Instructor

Matthew Berger

Office: Sony building, Rm 4028

Office hours: TF 10:00-11:00

MW 3:35-4:50pm, FGH 129

Go here for the syllabus.

Week 1 (August 19): Course introduction, ML basics

Wednesday: Course introduction, review on regression slides, build slides

Reading: MML Ch. 6.1 - 6.4, Ch. 8.1 - 8.2, Ch. 9.1 - 9.2, FML App. C

Week 2 (August 26): Gradient descent

Monday: Gradient descent slides, build slides	Reading: MML Ch. 5, Ch. 7.1, FML App. A, B
Wednesday: Stochastic gradient descent slides, build slides, Function approximation notebook	Reading: LSML Sec. 2 - 4
Thursday: — Assignment 1 Posted

Week 3 (September 2): Topics on gradient descent

Monday: Noise reduction slides, build slides	Reading: LSML Sec. 5 - 7
Wednesday: Neural tangent kernel slides, build slides	Reading: MML Ch. 12.4, Neural tangent kernel, Fourier features, Tangent space task arithmetic

Week 4 (September 9): Bayesian inference: basics, parametric and nonparametric methods

Monday: Bayesian statistics, linear regression slides, build slides	Reading: PML-2 Ch. 2.3, Ch. 3.2, Ch. 15.2.1 - 15.2.4
Wednesday: Gaussian processes slides, build slides	Reading: GP Ch. 2, PML-2 Ch. 3.7 - 3.8
Friday: — Assignment 1 Due

Week 5 (September 16): Approximate inference, variational inference

Monday: Model selection, Laplace approximation, information theory basics, ELBO slides, build slides — Assignment 2 Posted	Reading: PML-1 Ch. 6.1 - 6.2, PML-2 Ch. 5.1, Ch. 7.4, Ch. 10.1
Wednesday: Gradient-based variational inference slides, build slides	Reading: PML-2 Ch. 6.3.5, Ch. 10.2, MCGE

Week 6 (September 23): Variational inference, MCMC

Monday: Variational inference continued slides, build slides	Reading: PML-2 Ch. 6.3.5, Ch. 10.2, MCGE
Wednesday: Monte Carlo, Markov chains slides, build slides	Reading: PML-2 Ch. 11.1 - 11.4, Ch. 12.1 - 12.2
Friday: — Assignment 2 Due

Week 7 (September 30): Project introductions, midterm

Monday: Project introductions
Wednesday: Midterm

Week 8 (October 7 slides, build slides): MCMC

Monday: MCMC: Metropolis-Hastings, Gibbs sampling, mixture models	Reading: PML-2 Ch. 12.1 - 12.3, 12.5
Wednesday: Gibbs sampling continued, HMC slides, build slides	Reading: PML-2 Ch. 17.1 - 17.4

Week 9 (October 14): Bayesian neural networks, domain adaptation

Monday: Bayesian neural networks slides, build slides — Assignment 3 Posted	Reading: PML-2 Ch. 17.1 - 17.4
Wednesday: Out-of-distribution detection, domain adaptation slides, build slides	Reading: PML-2 Ch. 19.1 - 19.7

Week 10 (October 21): Active learning, few-shot learning

Monday: (Bayesian) Active learning slides, build slides	Reading: (PML-2) Ch. 34.7, Bayesian active learning, Deep Bayesian Active Learning
Wednesday: Few-shot learning slides, build slides	Reading: MAML, Learning to compare, ANIL

Week 11 (October 28): Generative models: variational autoencoders

Monday: Variational autoencoders: basics, posterior collapse slides, build slides — Assignment 3 Due	Reading: (PML-2) Ch. 21.1 - 21.2, 21.4
Wednesday: Variational autoencoders: disentanglement, discrete latents slides, build slides	Reading: (PML-2) Ch. 21.3, 21.5 - 21.6

Week 12 (November 4): Generative models: normalizing flows

Monday: Normalizing flows: basics, discrete flows slides, build slides	Reading: (PML-2) Ch. 23
Wednesday: Normalizing flows: continuous-time flows slides, build slides	Reading: (PML-2) Ch. 23
Friday: — Project midway report Due

Week 13 (November 11): Generative models: score matching, diffusion models

Monday: Energy-based models slides, build slides	Reading: (PML-2) Ch. 24
Wednesday: Diffusion models slides, build slides	Reading: (PML-2) Ch. 25

Week 14 (November 18): Unsupervised learning

Monday: Latent factor analysis slides, build slides
Wednesday: Project discussions

Week 15 (November 25): Thanksgiving break

Week 16 (December 2): Project presentations

Week 17 (December 9): Final project submissions

Monday:
— Project final submission Due