CMPUT 652: Reinforcement Learning with Robots (Fall 2019)


DXL-Tracker UR-Reacher-6 Create-Docker


[Schedule] (Contains slides and course materials)

Overview

In this new course, we will study how reinforcement learning (RL) algorithms can be used to learn to control physical robots in real-time. One of the main goals of RL agents is to learn to solve a given task by interacting with an unknown, unstructured environment. Due to simple and intuitive foundations and a series of breakthroughs in computer and board games, RL has become one of the most appealing branches of artificial intelligence. Recent RL methods have also shown excellent results in controlling robots in the virtual world. Now RL is viewed as a promising approach for controlling and adapting to unstructured environments with physical robots. To what extent are current methods capable of achieving this goal?

We will study the foundations of RL to be able to develop policy learning methods and learn about systematic ways of studying a real-time system to reveal the uncertainties involved in real-world tasks. This investigation will allow us to understand the differences between real-world and standard simulated tasks so that we can adapt task setups and algorithmic implementations to the real world as well as enhance the simulated tasks to incorporate the additional challenges in real-time systems. En route, we will learn about other promising approaches to learning in robotics that are not performed in real-time, such as learning from demonstration and simulation-to-reality transfer.

Background in machine learning, programming, or robotics would be essential in this course.

Objectives

Prerequisites

This course requires knowledge in basic probability theory and programming ability using deep learning frameworks such as PyTorch in Python 3.

Course Materials

The book by Sutton and Barto (2018) will be used as the main textbook. Apart from that, we will rely on scientific papers, which will be provided in the course schedule.

Course Topics

Course work and evaluation