csc412

CSC 412/2506 Winter 2025

Probabilistic Machine Learning

This course introduces probabilistic learning tools such as exponential families, directed graphical models, Markov random fields, exact inference techniques, message passing, sampling and mcmc, hidden Markov models, variational inference, Neural networks, Embeddings, Attention, Sparse Autoencoders, MoE - Mixture of Experts, Contrained Decoding, Speculative Decoding, Variational autoencoders, and Diffusion models. It will also offer a broad view of model-building and optimization techniques that are based on probabilistic building blocks which will serve as a foundation for more advanced machine learning courses.

More details can be found in syllabus and piazza.

---

Announcements:

• Practice final Answers is released!
• Practice final is released!
• Assignment 4 is released! Due on 03/30/ 22:59 pm (11pm EST)
• Assignment 3 is released! Due on 03/16/ 22:59 pm (11pm EST)
• A2 - To provide more time during reading week, we’ll remove the penalty for late submissions (up to 3 days)
• A2 has been released, due on 02/16/ 22:59 pm (11pm EST)
• A1 has been released! Due on 02/02 22:59 (11pm EST). TA office hours are 01/21 5-6pm, 01/28 5-6pm at BA2270.
• Lectures begin on Jan 6!

---

Instructors:

Instructor	Denys Linkov
Email	csc412prof@cs.toronto.edu
Office hours	TH 5-6 BA2272

---

Teaching Assistants:

Stephan Rabanser, Vahid Balazadeh, Kelly Zhu, Alireza Mousavi, Chen-Hao Chao, Jerry Ji, Alireza Keshavarzian

• Email: csc412ta@cs.toronto.edu

• TA office hours Tues 5-6pm, BA2270

Time & Location:

Section	Lecture	Tutorial
CSC412H1-S-LEC5101	M 6-8pm @ MP 134	M 8-9pm @ MP 134
CSC412H1-S-LEC52011	Th 6-8pm @ MP 134	Th 8-9pm @ MP 134

---

Suggested Reading

No required textbooks. Suggested reading will be posted after each lecture (See lectures below).

• (PRML) Christopher M. Bishop (2006) Pattern Recognition and Machine Learning
• (MLPP) Kevin P. Murphy (2012), Machine Learning: A Probabilistic Perspective
• (PML1) Kevin P. Murphy (2022), Probabilistic Machine Learning: An Introduction
• (PML2) Kevin P. Murphy (2023), Probabilistic Machine Learning: Advanced topics
• (ITIL) David MacKay (2003) Information Theory, Inference, and Learning Algorithms
• (ESL) Trevor Hastie, Robert Tibshirani, Jerome Friedman (2009) The Elements of Statistical Learning

---

Lectures and timeline

Week	Lectures	Recordings	Suggested reading	Tutorials	Timeline
1	Introduction Probabilistic Models	Lecture 1 on quercus	MLPP 1 & 2 PRML 2.4	tut01	syllabus
2	Decision theory Directed Graphical Models	Lecture 2 on quercus Tutorial 2 on quercus, Mon lecture	PRML 1.5 MLPP 10	tut02
3	Markov Random Fields Exact inference	Lecture 3 on quercus	MLPP 19-19.5, 20.3 ITIL 21.1, 26	tut03	A1 out
4	Message passing Monte Carlo Methods	Lecture 4 on quercus	MLPP 20.2,22.2 ITIL 29	tut04	A1 due
5	Sampling I Sampling II	Lecture 5 on quercus	MLPP 17.2, 24.3	tut5 notebook Sampling simulations	A2 out
6	Hidden Markov Models Variational inference I	Lecture 6 on quercus	MLPP 17.3 MLPP 21.1-3	colab Additional Examples: CSC486 SLP	A2 due
7	Reading week (No Class/Tutorial)	-	—	—	—
8	Midterm exam prep Practice exam Solutions				midterm
9	Variational inference II Variational Autoencoders		PRML 10.1-10.2 2015 Neurips Talk		A3 out
10	Embeddings Attention		tut10	Illustrated Transformer Bert Topic Finetuning a Classifier	A3 due
11	Sparse Autoencoders Constrained Decoding		AntropicSAEPaper OpenAISAEPaper OpenAISAE Training Code Gated SAE DOMINO Paper	Tokenizer colab SAE Colab	A4 out
12	Speculative Decoding Diffusion models		This blog CPVR tutorial Tutorial notes
13	Guest Lectures Final exam review				A4 Due

---

Assignments

Assignment #	Out	Due	TA Office Hours
Assignment 1	1/20	2/02	01/21 5-6pm, 01/28 5-6pm at BA2270
Assignment 2	2/03	2/16	Friday, Feb. 7, 10 - 11 AM and Tuesday, Feb. 11, 10 - 11 AM (Zoom), Wednesday, Feb. 12, 10 - 11 AM (Pratt Building, Room 286.)
Assignment 3	03/03	3/16	TBA
Assignment 4	03/17	3/30	TBA