Reaching the Limit in Autonomous Racing: Optimal Control versus Reinforcement Learning

Abstract

Autonomous racing is a challenging problem requiring planning and decision-making at the limits of vehicle handling. We present a comprehensive comparison of model-based optimal control (OC) and model-free reinforcement learning (RL) for autonomous drone racing across two scenarios: time-optimal racing and head-to-head racing. In time-optimal racing, OC achieves lower lap times in simulation due to accurate modeling, while RL transfers more robustly to the real world. In head-to-head racing, OC can compute optimal overtaking maneuvers while RL develops emergent defensive behaviors. Our study reveals complementary strengths of both paradigms and provides practical guidance for autonomous racing algorithm selection.

Resources

Video

Citation

@article{song2023reaching,
  title     = {Reaching the Limit in Autonomous Racing: Optimal Control versus Reinforcement Learning},
  author    = {Song, Yunlong and Romero, Angel and M{\"{u}}ller, Matthias and Koltun, Vladlen and Scaramuzza, Davide},
  journal   = {Science Robotics},
  volume    = {8},
  number    = {82},
  year      = {2023},
  doi       = {10.1126/scirobotics.adg1462}
}