MGNet: Monocular Geometric Scene Understanding for Autonomous Driving

This repository contains the official implementation of our ICCV 2021 paper MGNet: Monocular Geometric Scene Understanding for Autonomous Driving.

This is a re-implementation based on detectron2, hence results differ slightly compared to the ones reported in the paper.

Installation

See INSTALL.md for instructions on how to prepare your environment to use MGNet.

Usage

See datasets/README.md for instructions on how to prepare datasets for MGNet.

See GETTING_STARTED.md for instructions on how to train and evaluate models, or run inference on demo images.

See trt_inference/README.md for instructions on how to export trained models to TensorRT and run optimized inference.

Model Zoo

All models were trained using 4 NVIDIA 2080Ti GPUs.

Name	PQ	PQ_St	PQ_Th	Abs Rel	RMSE	δ < 1.25	Download
MGNet Cityscapes Fine	54.879	62.524	44.367	0.188	8.439	0.744	model
MGNet Cityscapes Video Sequence	55.644	63.140	45.337	0.166	7.984	0.794	model
MGNet KITTI Eigen Zhou	-	-	-	0.095	3.788	0.897	model

Reference

Please use the following citations when referencing our work:

MGNet: Monocular Geometric Scene Understanding for Autonomous Driving (ICCV 2021)
Markus Schön, Michael Buchholz and Klaus Dietmayer, [paper], [video]

@InProceedings{Schoen_2021_ICCV,
    author    = {Sch{\"o}n, Markus and Buchholz, Michael and Dietmayer, Klaus},
    title     = {MGNet: Monocular Geometric Scene Understanding for Autonomous Driving},
    booktitle = {Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)},
    month     = {October},
    year      = {2021},
    pages     = {15804-15815}
}

Acknowledgement

We used and modified code parts from other open source projects, we especially like to thank the authors of:

• detectron2
• packnet-sfm
• TorchSeg