IE 498: Online Learning and Decision Making

Fall 2019

Tuesday, Thursday 3:30pm - 4:50pm, TB 114

Instructor: Yuan Zhou.
Teaching Assistant: Tanmay Gangwani.


This course covers several foundational topics in online learning and sequential decision making under uncertainty, a subject on the intersection of algorithms, machine learning, and operations research. Such problems have wide applications in online advertising, recommendation systems, crowdsourcing, revenue management, etc. In this course, we will study the problems that usually feature the tension between how to collect data and utilize the data to make optimal sequential decisions (a.k.a. the exploration and exploitation dilemma). We cover both fundamental results and research frontiers. We focus on algorithmic results, and introduce lower bounds as well.


This course assumes background in basic probability theory. linear algebra, and algorithm design and analysis at an undergraduate level. Key mathematical concepts will be reviewed before they are used, but a certain level of mathematical maturity is expected.

Lecture Notes

Disclaimer: The lecture notes are scribed by the students from the class, and have not been proofread by the lecturer.