|
AI543: Foundations of Online Machine Learning – Winter 2026
Course Description
In this course, we will focus on algorithms for online learning and sequential decision-making including online convex optimization, bandits, and online reinforcement learning, examine their theoretical guarantees and applications in real-world machine learning problems such as information retrieval (advertising, recommender system, web search and ranking).
Course Information
Lectures
Tuesday & Thursday 12 - 1:50pm, Bexell Hall 412
Instructor
Huazheng Wang
Email: huazheng.wang [at] oregonstate.edu
Office: KEC 3097
Office hours: Thursday 2-4pm, KEC 3097
Prerequisites
AI 534 or CS 534.
Contact
We will use Canvas for slides and assignments, and Discord for communication. See Canvas announcements for the link to Discord channel.
Prerequisites
Familiar with probability, statistics, linear algebra, calculus and machine learning.
Python. We will use Python for programming assignments.
Schedule
| Week | Date | Lecture | Readings | Notes |
| Week 1 | 1/6 | Introduction to the course | | |
| 1/8 | Review of linear algebra, statistics and optimization | |
| Week 2 | 1/13 | Online Gradient Descent | | |
| 1/15 | Online-to-Batch Conversion | | HW1 posted. |
| Week 3 | 1/20 | Follow the Regularized Leader | | |
| 1/22 | Online Learning with Expert Advice | | |
| Week 4 | 1/27 | Stochastic Multi-Armed Bandits | | |
| 1/29 | Regret Lower Bounds | | HW1 due. HW2 posted. |
| Week 5 | 2/3 | Multi-Armed Bandits and Linear Bandits | | |
| 2/5 | Thompson Sampling | | |
| Week 6 | 2/10 | Generalized Linear Model, Kernel, and Neural Bandits | | |
| 2/12 | Bandit Convex Optimization | | HW2 due. HW3 posted. |
| Week 7 | 2/17 | Contextual Bandits | | |
| 2/19 | Non-stationary Regret Minimization | | |
| Week 8 | 2/24 | Online Learning to Rank | | |
| 2/26 | Combinatorial Bandits Learning | | HW3 due. HW4 posted. |
| Week 9 | 3/3 | Vulnerability and Robustness of Online Learning | | |
| 3/5 | Online Reinforcement Learning | | |
| Week 10 | 3/10 | Project presentations | | |
| 3/12 | Project presentations | | HW4 due. |
|
|
Gradings
Resources
Suggested readings:
A Modern Introduction to Online Learning, Francesco Orabona.
Bandit Algorithms by Tor Lattimore and Csaba Szepesvári
|