Simultaneous Localization and Mapping algorithms for Formula Student Driverless @Technion

TODO:

• add detailed description

See demos

See wiki

Table of Contents

• Installation
• Roadmap
  • Visual Cone Detection
  • Visual SLAM Exploration
  • Visual SLAM System
  • LIDAR Setup
  • LIDAR Cone Detection
  • LIDAR SLAM Exploration
  • LIDAR Landmark Based SLAM
  • Model Predictive Control
  • System
  • Simulator
  • Hardware

Installation

See wiki/Installation

Roadmap

Currently working on bold

Visual Cone Detection

• annotate real data
  • see instructions
• train YOLOv3-tiny
• architecture search, see yolo_cone_detection.md
• rewrite with TensorRT
• test inference performance on Xavier
• setup AirSim - next project
  • add cone position information
  • dataset generation pipeline
  • augmentation
    • weather - built in
    • noise
    • low light
  • generate dataset
• improvements:
  • add sub-pixel refinement
• add tracking
  • implement KLT - works OK
  • test opencv multitracker - poor
  • implement ORB cone descriptor - need to test
  • test Tracking-with-darkflow

Visual SLAM Exploration

• test orbslam 2
  • discarded as not robust enough
• test orbslam_dwo
  • discarded since no RT branch available, also slower than original orbslam
• test rovio
  • works OK on servo data
• test servo on provided data
  • bugfixes - doesn't compile
  • compile
  • test ROS node on provided data - works fine
• conclusion: continue working with servo
• need to check:
  • VIORB
  • ORB-YGZ-SLAM

Visual SLAM System

• integrate servo with the main system
• calibrate our cameras and IMU
• test on our data
• performance optimization
  • integrate with orbslam2-gpu
  • add binary vocabulary
  • remove Pangolin GUI
• publish points and poses to ROS
• test cone detection
• implement cone detector and descriptor
• implement data fusion in orbslam
• test on the car

LIDAR Setup

• install ROS velodyne drivers
• test and store data
• control LIDAR speed from ROS, see here
• filter LIDAR by FOV and distance, search for ring information, see here
• LIDAR-camera calibration
  • lidar_camera_calibration doesn't compile, todo: debug
  • try this
  • velo2cam_calibration
• fusion with camera

LIDAR Cone Detection

• test the data
• add node to ROS
• ground removal
• cone segmentation and clustering
• cone tracking
  • probably redundant
• additional filtering
• color from intensity
• color from camera
• improvements
  • SqueezeSeg

LIDAR SLAM Exploration

• test off the shelf LIDAR SLAM solutions
  • loam_velodyne - not good enough for tracking cones
  • hdl_graph_slam - doesn't compile
  • LeGO-LOAM - worse than loam_velodyne
  • cartographer - compiled, looks promising, but integration is difficult
• next will unlikely work
  • mrpt
  • gmapping
  • hector_mapping - similar to gmapping
• conclusion: general-purpose LIDAR SLAM is not good enough

LIDAR Landmark Based SLAM

• explore directions
  • Kalman Filter vs Graph Optimization
  • g2o vs gtsam

System

• define ROS node topology
• add documentation

Simulator

• eufs_sim - gazebo simulator
  • compiled
  • bugfix - no car
• AirSim - Microsoft's framework
  • test on linux
  • add lidar, see here
  • add IMU, see here
  • integrate with ROS, see here
• fssim - a new one

Hardware

• camera - ZED
• LIDAR - Velodyne VLP-16
• Nvidia Jetson setup
• capture with rosbag
• install and run on the car
• capture demo content
• capture real content
• Nvidia Drive PX2 setup
• after meeting with Nvidia decided to use Jetson AGX Xavier
• improve capture
  • find better camera + IMU, see camera considerations
  • debug low fps
• automate capture - launch files
• capture new content
• Nvidia Jetson Xavier setup
• install and run on the car
• TODO: update regarding the electric car
  • sensors
  • control

Model Predictive Control

• explore different options:
  • MPCC Model Predictive Contouring Controller for Autonomous Racing
  • MPPI Model Predictive Path Integral Controller
• start working on simulator