Champion-level Drone Racing using Deep Reinforcement Learning

Abstract

Autonomous systems that exceed human performance in specific domains have become paradigm-defining achievements. Racing drones are a particularly challenging domain: at speeds above 100 km/h, expert pilots are among the world's best at real-time decision making. In this work we show that a trained neural network can beat human world champions in a real-world competitive race. Our AI system, Swift, combines deep reinforcement learning trained in simulation with zero-shot real-world deployment, using visual and inertial sensor inputs to operate at up to 108 km/h in a real racing arena. In races against three human champions, Swift won 15 out of 25 races, and its fastest lap was 0.5 seconds quicker than the best human time. Our work demonstrates that an autonomous system can achieve superhuman performance in an extreme real-world sport.

Resources

Video

Citation

@article{kaufmann2023champion,
  title     = {Champion-level Drone Racing using Deep Reinforcement Learning},
  author    = {Kaufmann, Elia and Bauersfeld, Leonard and Loquercio, Antonio and M{\"{u}}ller, Matthias and Koltun, Vladlen and Scaramuzza, Davide},
  journal   = {Nature},
  volume    = {620},
  pages     = {982--987},
  year      = {2023},
  doi       = {10.1038/s41586-023-06419-4}
}