🤖 Modern Robot Learning 🦾

Hands-on Tutorial

This course provides a practical introduction to training robots using data-driven methods. Key topics include data collection methods for robotics, policy training methods, and using simulated environments for robot learning. Throughout the course, students will have hands-on experience to design simulation environments, collect data, train policies, and evaluate policies. A solid working knowledge of Python and a basic understanding of machine learning are prerequisites. The course focuses entirely on the project, with no additional assignments.

Register Now

Overview

Robot Data Collection

People say robotics is all about data, but how do we collect it? Learn and experience how the community is collecting data for robots.

Policy Training

With the data collected, how do we train a policy to perform the task autonomusly? Learn various policy training methods commonly used in the community.

Simulation for Robotics

What role does simulation play in robotics? Learn how to model your environment in simulation and transfer your learned policy to the real world.

Course Schedule

Classes held in Stata 32-124 / Office hours held in 45-322

January 10 (Fri): Course Overview Slide Video

Robot Learning Basics — Manipulation Focused

  • Paradigm Shift in Robotics: Learning from Data
  • Data Collection Methods

January 13 (Mon): Data Collection & Policy Training Slide Video

  • Collecting Robot Data in Real-World: Robot Teleoperation
  • Policy Training Techniques
  • Simulation for Robotics: Real2Sim and Sim2Real

January 15 (Wed): Alternative Data Sources for Robotics Slide Video

  • Alternatives to Teleoperated Datasets: Wearable Devices and Human Videos
  • Challenges and Opportunities in Policy Training

January 17 (Fri): Project Overview

  • Hands-on Tutorial Guidelines
  • Robot Data Collection in Virtual Reality

Hands-on Tutorial

After the lectures, you'll have the opportunity to apply what you've learned through hands-on experimentation. The remainder of the IAP course following January 17 will consist of office hours and independent project work. We will announce the detailed office hours schedule soon.

Data Collection

Reserve Slot →

Gather robot manipulation data through teleoperation with virtual reality interfaces like Apple Vision Pro.

Policy Training

View Code

Train robot policies using behavioral cloning and reinforcement learning techniques

Simulation Evaluation

Test and evaluate trained policies in realistic simulation environments

Sim2Real Transfer

Optional opportunity to deploy trained policies on real robot hardware

Course Instructors

Instructor

Haoshu Fang

MIT CSAIL, Post-Doc

Instructor

Younghyo Park

MIT CSAIL, PhD Student

Instructor

Jagdeep Bhatia

MIT CSAIL, MEng Student

Instructor

Lars Ankile

MIT CSAIL, Visiting Researcher