AI539: Introduction to Online Learning – Fall 2023

Course Description

In this course, we will focus on algorithms for online learning and sequential decision-making including online convex optimization and bandits, examine their theoretical guarantees, and applications in real-world machine learning problems such as information retrieval (advertising, recommender system, web search and ranking).

Course Information

Lectures

Tuesday & Thursday 2 - 3:50pm, Strand Agriculture Hall 113

Instructor

Huazheng Wang
Email: huazheng.wang [at] oregonstate.edu
Office: KEC 3097
Office hours: Thursday 4 - 6pm

Contact

We will use Canvas for slides and assignments, and Discord for communication. See Canvas announcements for the link to Discord channel.

Prerequisites

Familiar with probability, statistics, linear algebra, calculus and machine learning.
Python. We will use Python for programming assignments.

Schedule

Week	Date	Lecture	Readings	Notes
Week 1	9/28	Introduction to the course
Week 2	10/3	Review of linear algebra, statistics and optimization
	10/5	Online Gradient Descent
Week 3	10/10	Online-to-Batch Conversion		HW1 posted.
	10/12	Follow the Regularized Leader
Week 4	10/17	Online Learning with Expert Advice
	10/19	Stochastic Multi-Armed Bandits
Week 5	10/24	Regret Lower Bounds		HW1 due. HW2 posted.
	10/26	Multi-Armed Bandits and Linear Bandits
Week 6	10/31	Thompson Sampling
	11/2	Generalized Linear Model, Kernel, and Neural Bandits
Week 7	11/7	Bandit Convex Optimization		HW2 due. HW3 posted.
	11/9	Contextual Bandits
Week 8	11/14	Non-stationary Regret Minimization
	11/16	Online Learning to Rank
Week 9	11/21	Combinatorial Bandits Learning		HW3 due. HW4 posted.
	11/23	Thanks giving
Week 10	11/28	Vulnerability and Robustness of Online Learning
	11/30	Online Reinforcement Learning
Week 11	12/5	Project presentations		HW4 due.
	12/7	Project presentations

Gradings

Homework – (3*20%) 60%
Final project – (proposal 5%, presentation 15%, report 20%) 40%
Total – 100%