Driving Policy Transfer via Modularity and Abstraction

Abstract

Driving policies trained in simulation must bridge the sim-to-real gap when deployed on real vehicles. We propose a modular approach that decouples the driving policy from the visual perception system. The driving policy is trained entirely in simulation using an idealized semantic perception module that provides structured representations such as waypoints and semantic maps. At deployment, this module is replaced by a real-world perception system without any policy retraining. Experiments in the CARLA simulator and on a real vehicle demonstrate that our modular approach substantially outperforms end-to-end methods and is significantly more robust to the visual domain shift from simulation to the real world.

Resources

Citation

@inproceedings{mueller2018driving,
  title     = {Driving Policy Transfer via Modularity and Abstraction},
  author    = {M{\"{u}}ller, Matthias and Dosovitskiy, Alexey and Ghanem, Bernard and Koltun, Vladlen},
  booktitle = {Conference on Robot Learning (CoRL)},
  year      = {2018}
}