Teaching UAVs to Race: End-to-End Regression of Agile Controls in Simulation

Abstract

We present an approach to autonomous UAV racing using end-to-end deep learning trained in a photo-realistic simulator. Our network directly regresses agile flight control commands from raw first-person-view (FPV) images. We train using a teacher-student approach with a privileged teacher that has access to the UAV's full state, generating smooth training trajectories that are then imitated by the student from raw image inputs. The system learns to navigate a racing course at high speed without requiring any manual feature engineering. The work received the Best Paper Award at the ECCV 2018 UAVision workshop.

Resources

Citation

@inproceedings{mueller2018teaching,
  title     = {Teaching {UAVs} to Race: End-to-End Regression of Agile Controls in Simulation},
  author    = {M{\"{u}}ller, Matthias and Casser, Vincent and Smith, Neil and Michels, Dominik L. and Ghanem, Bernard},
  booktitle = {ECCV Workshops (UAVision)},
  note      = {Best Paper Award},
  year      = {2018}
}