Skip to content

metadriverse/SimGen

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

8 Commits
 
 
 
 

Repository files navigation

SimGen: Simulator-conditioned Driving Scene Generation

Revive driving scene simulation by simulator-conditioned generative models

SimGen: v1.0 License: Apache2.0

Yunsong Zhou, Michael Simon, Zhenghao Peng, Sicheng Mo, Hongzi Zhu, Minyi Guo, and Bolei Zhou

Highlights

🔥 The first simulator-conditioned generative model for controllable driving scene generation with appearance and layout diversity.

🌟 SimGen addresses simulation to reality (Sim2Real) gaps via cascade diffusion paradigm, and follows layout guidance from simulators and cues of the rich text prompts to realistic driving scenarios.

method

📊 DIVA dataset comprises 147.5 hours of web videos and synthesized data for diverse scene generation and advancing Sim2Real research.

News

  • [2024/06] SimGem paper released.
  • [2024/06] DIVA dataset subset released.

Table of Contents

  1. Highlights
  2. News
  3. TODO List
  4. DIVA Dataset
  5. License and Citation
  6. Related Resources

TODO List

  • Release DIVA dataset
  • Release SimGen code
  • Toolkits for novel scene generation

DIVA Dataset

method

DIVA-Real. It collects driving videos from YouTube, covering a worldwide range of geography, weather, scenes, and traffic elements and preserving the appearance diversity of a wide range of traffic participants. Here we provide a sample of 🔗 YouTube video list we used. For privacy considerations, we are temporarily keeping the complete data labels private.

method

DIVA-Sim. The Sim2Real data is induced from the same real-world scenarios, in which we can obtain real-world map topology, layout, and raw sensor data. It also includes hazardous driving behaviors through interactions introduced by adversarial traffic generation. The digital twins (on nuScenes dataset) and safety-critical scenarios (on Waymo Open dataset) can be obtained through this 🔗data link.

License and Citation

All assets and code in this repository are under the Apache 2.0 license unless specified otherwise. The annotation data is under CC BY-NC-SA 4.0. Other datasets (including nuScenes, Waymo, and MetaDrive) inherit their own distribution licenses. Please consider citing our paper and project if they help your research.

@article{zhou2024simgen,
  title={SimGen: Simulator-conditioned Driving Scene Generation},
  author={Zhou, Yunsong and Simon, Michael and Peng, Zhenghao and Mo, Sicheng and Zhu, Hongzi and Guo, Minyi and Zhou, Bolei},
  journal={arXiv preprint arXiv:2406.09386},
  year={2024}
}

Related Resources

We acknowledge all the open-source contributors for the following projects to make this work possible:

You are welcome to follow other related work from Twitter Follow, MetaDriverse, and GenForce.