Sim4CV: A Photo-Realistic Simulator for Computer Vision Applications

Abstract

We present Sim4CV, a photo-realistic simulator for computer vision applications built on Unreal Engine 4. Sim4CV enables rapid generation of large amounts of synthetic training data with accurate ground truth for tasks such as visual tracking, depth estimation, and object detection. We demonstrate its utility on UAV tracking: we generate a large-scale synthetic UAV tracking dataset, use it to train deep tracking models, and show that synthetic pretraining significantly improves performance on real-world UAV tracking benchmarks. Sim4CV supports arbitrary scene composition, day-night cycles, weather conditions, and flexible camera rigs, making it a general-purpose tool for the computer vision community.

Resources

Citation

@article{mueller2018sim4cv,
  title     = {{Sim4CV}: A Photo-Realistic Simulator for Computer Vision Applications},
  author    = {M{\"{u}}ller, Matthias and Casser, Vincent and Lahoud, Jean and Smith, Neil and Ghanem, Bernard},
  journal   = {International Journal of Computer Vision (IJCV)},
  year      = {2018}
}