This course covers several foundational topics in online learning and sequential decision making under uncertainty, a subject on the intersection of algorithms, machine learning, and operations research. Such problems have wide applications in online advertising, recommendation systems, crowdsourcing, revenue management, etc. In this course, we will study the problems that usually feature the tension between how to collect data and utilize the data to make optimal sequential decisions (a.k.a. the exploration and exploitation dilemma). We cover both fundamental results and research frontiers. We focus on algorithmic results, and introduce lower bounds as well.
This course assumes background in basic probability theory. linear algebra, and algorithm design and analysis at an undergraduate level. Key mathematical concepts will be reviewed before they are used, but a certain level of mathematical maturity is expected.
Disclaimer: The lecture notes are scribed by the students from the class, and have not been proofread by the lecturer.