Link back to the Syllabus

All readings are from the textbook. These readings are designed to be short, so that it should be easy to keep up with the readings.

For course material from week 11 till the end, see eclass

Office Hours:

See eClass

Note: All the office hours will be conducted over video chat. Links are posted on eclass.

Schedule:

This schedule is tentative, and is likely to change throughout the semester.

We will refer to Coursera courses as mini-courses and weeks within a mini-course as modules. Therefore, Markov Decision Processes, which is under week 2 of Coursera Course 1: Fundamentals of Reinforcement Learning will be referred to as mini-course 1, module 2.

Week	Date	Topic	Deadlines
1	January 6, 8, 10	Introduction (Jan 6) Worksheet in class about random variables and expectations Probabilities & expectations Probabilities slides (Jan 10)	In preparation for Week 2, watch videos in mini-course 1, module 1, starting with the K-Armed Bandit
2	January 13, 15, 17	Background on Statistics and Probability & the K-Armed Bandit Problem Probabilities and Expectations review (Jan 13) Multi-armed Bandits (Jan 15) Worksheet about Bandit Problem	Complete Practice Quiz of mini-course 1, module 1 by midnight on Sunday, January 12 link Complete Graded Notebook of mini-course 1, module 1 by midnight on Friday, January 17, link
3	January 20, 22, 24	Markov Decision Processes MDP review and examples (Jan 20) More MDP review and examples (Jan 22) More MDP review and examples (Jan 24) Worksheet about Markov Decision Processes	Complete Practice Quiz of mini-course 1, module 2 by midnight on Sunday, January 19 link Complete Graded Assignment Submission of mini-course 1, module 2 by midnight on Thursday, January 23, link Complete Graded Assignment Peer Review of mini-course 1, module 2 by midnight on Sunday, January 26, link
4	January 27, 29, 31	Value Function and Bellman Equations Jan 27 Jan 29 Jan 31	Complete Practice Quiz of mini-course 1, module 3 by midnight on Sunday, January 26 link Complete Graded Quiz of mini-course 1, module 3 by midnight on Friday, January 31, link
5	February 3, 5, 7	Dynamic Programming Feb 3 Feb 5 Feb 7 Worksheet about Dynamic Programming	March 13 last day to drop course (50% fees) Complete Practice Quiz of mini-course 1, module 4 by midnight on Sunday, Feburary 2 link Complete Graded Notebook of mini-course 1, module 4 by midnight on Friday, Feburary 7, link
6	February 10, 12, 14	Monte Carlo Methods Feb 10 Feb 12 Feb 14	Complete Practice Notebook of mini-course 2, module 1 by midnight on Sunday, Feburary 9, link Complete Graded Quiz of mini-course 2, module 1 by midnight on Friday, Feburary 14, link
7	February 17, 19, 21	No classes: Reading week
8	February 24, 26, 28	Temporal Difference Methods for prediction Feb 24 Feb 26 Feb 28	Complete Practice Quiz of mini-course 2, module 2 by midnight on Sunday, Feburary 23, link Complete Graded Notebook of mini-course 2, module 2 by midnight on Friday, Feburary 28, link
9	March 2, 4, 6	Temporal Difference Methods for control Mar 2 Mar 4 Mar 6	Complete Practice Quiz of mini-course 2, module 3 by midnight on Sunday, March 1, link Complete Graded Notebook of mini-course 2, module 3 by midnight on Friday, March 6, link
10	March 9, 11, 13	Planning, Learning and Acting Mar 9 Mar 11	Complete Practice Quiz of mini-course 2, module 4 by midnight on Sunday, March 8, link Complete Graded Notebook of mini-course 2, module 4 by midnight on Friday, March 13, link
11	March 16, 18, 20	Midterm: review, practice and exam	Midterm on March 20
12	March 23, 25, 27	Prediction with Approximation	Complete Practice Quiz of mini-course 3, module 1 by midnight on Sunday, March 22, link Complete Graded Notebook of mini-course 3, module 1 by midnight on Friday, March 27, link
13	March 30, April 1, 3	Constructing Features for Prediction & Control with Approximation	Complete Practice Quiz of mini-course 3, module 2 by midnight on Sunday, March 29, link Complete Graded Notebook of mini-course 3, module 2 by midnight on Friday, April 3, link
14	April 6, 8	Control with Approximation & Course Review	Complete Practice Quiz of mini-course 3, module 3 by midnight on Sunday, April 5, link Complete Graded Notebook of mini-course 3, module 3 by midnight on Friday, April 10, link Complete Capstone Project of mini-course 4 by midnight on Sunday, April 19. It consists of 2 Graded Quizzes and 2 Graded Notebooks (quiz1, quiz2, notebook1, notebook2)
Final	April 20 at 9:00 a.m. (tentative) Check Bear Tracks to ensure	Final Exam	You can bring a two page cheat-sheet