Machine Learning in Dynamic Environment

This course introduces online learning, a domain of machine learning concerned with algorithms that learn adaptively from a continuous stream of data. It is particularly relevant when environments are dynamic, and batch-wise learning is expensive or not scalable. Recommender systems are a prime example: when a new user signs up, the system has no prior knowledge of that user and must improve its recommendations on-the-fly by observing how the user responds. The goal in online learning is to make a sequence of accurate predictions given correct answers to previous instances, which are assumed to be observable after each decision.

The course introduces key techniques including multi-arm bandit learning, linear stochastic bandit learning, and online convex optimization, and examines their application to problems in recommender systems, economics, and other domains where prediction accuracy directly affects long-term platform performance and user engagement.

Have you considered how Netflix recommends movies to you? Or how you are  recommended items to buy on Amazon? Recommender systems are systems that  recommend restaurants, movies, or content to watch, etc., by learning user's preferences.  When a new user signs up, the system has no prior knowledge of the user and must improve  its recommendations on-the-fly by observing how the user responds. Such a paradigm of  machine learning where the system must learn "on-the-go" or “adapt” is termed online  learning. The goal in online learning is to make a sequence of accurate predictions given the correct answers to the previous instances, which is assumed to be observable post the  decision. This is reasonable for example in recommender systems where the platform can  observe whether the user has viewed a recommendation or not. The effectiveness of the  prediction, for instance in recommendations, is critical to long term engagement of the  users and the success of the platforms. It is particularly relevant where the users  themselves can be dynamic and the standard machine learning approach of batch updating  can be expensive in terms of performance and scaling.

The course will introduce some key techniques like multi-arm bandit learning, linear  stochastic bandit learning, online convex optimization, etc., and discuss their application to  many practically relevant problems in domains such as recommender systems, economics,  etc.

The course is an introductory level course aimed at introducing students to the basic  algorithmic techniques underlying online or adaptive learning.

• Understand the elements of algorithm design for learning in dynamic and/or online  scenarios.
• Apply the algorithms for problem solving
• Synthesize algorithms for learning in online/dynamic scenarios.

After completing this course, students should be able to:

• Describe/explain the various algorithms for machine learning in dynamic or online  scenarios| Know/Knowledge Outcome  
• Describe/explain how to synthesize algorithms | Know/Knowledge Outcome 
• Demonstrate problem solving through algorithm design | Comprehend Outcome 
• Create and synthesize algorithms by combining the various elements and identifying the right combination of various elements| Create/synthesize  Outcome  

1. Shai Shalev-Shwartz. Online Learning and Online Convex Optimization.
2. Elad Hazan. Introduction to Online Convex Optimization.
3. Tor Lattimore and Csaba Szepeswari, Bandit Algorithms.
4. Richard Sutton and Andrew Barto, Reinforcement Learning.

Info not available