This is the repository for our submission to Special Issue "Advanced Sensing Techniques for Autonomous Vehicles and Advanced Driver Assistance Systems (ADAS)".
The source code freeze at our submission can be found in branch hrnet_sensors_submit and deeplab_sensors_submit.
Our anonymized rosbag data, mapping results for different ablation studies and groundtruth labels are available in Google Drive. Additonal mapping results are also shared.
To run the experiment, we encourage use the docker file we provided to build an environment that allows hrnet and deeplab and their ROS wrappers to work properly. Build the base-cuda docker image first, them semantic-mapping docker image will depend on it.
In additional to code in this repository, we will also need pcl_ros and map_reduction are also used. The former loads a point cloud map and the latter crops the point cloud map based on localization information. Download the point cloud map from the Google Drive PointCloudMap folder.
rosrun pcl_ros pcd_to_pointcloud /path/to/point_cloud_map.pcd 0.1 _frame_id:=map /cloud_pcd:=/points_map
TODO:
- open-source code
- refactor code
- make data available (anonymized rosbag)
- make result map and ground truth available
- user guide
- docker file
- add sample results
The below guide is the initial version and will be updated soon.
We will fuse the LiDAR point cloud with the semantic segmented 2D camera image together and create a bird's-eye-view semantic map of the environment. We are developing this repository to adapt new sensor suits and new software architectures.
Refer to this documentation of how to setup the development environment.
There are still some none trivial step for you to run our experiment presented in the paper, and that is why we are here to guide you.
To run the whole map generation, you should first set the Start Time
in the Simulation Tab of the Autoware to 410
. Then create an configuration file, and have the following keys set to
TEST_END_TIME: 1581541450
MAPPING:
BOUNDARY: [[0, 1000], [0, 1400]]
RESOLUTION: 0.2
To run a small region of the map for testing, you should first set the Start Time
to 390
. Then use the following configuration
TEST_END_TIME: 1581541270 # Which is about 20s after 390s.
MAPPING:
BOUNDARY: [[100, 300], [800, 1000]]
RESOLUTION: 0.2 # This one doesn't really matter
Then pass these configuration into the roslaunch as shown in Step 4 and click the Play
in the Autoware, now you should be good to go!
To reproduce our result, you need to
- Slow down the replay speed of the rosbag. As the image node will drop package if it is legging behind, the drop of package is uncontrollable and can happen at anytime. Therefore we need to slow down the replay speed to ensure that all the packages are processed. We recommend using
0.1
replay rate. - Set the input image scale to 1.0. The nearest neighbor interpolation from the
cv2
may cause some misalignments and we want to disable that. - Fix the random seed. Set the
RNG_SEED
value in the configuration file to a non zero value so that we can disable any randomness.
You can install rospy via pip by (Reference is here)
pip install --extra-index-url https://rospypi.github.io/simple/ rospy
We have provided a script install_ros_python_packages.sh
in the script folder. You can run it by
bash ./install_ros_python_packages.sh
In order for your Python script to be detectable by the catkin_make
, you have to make it executable
. Actually, catkin_make
will collect all the executable files in your source code and make them visible in rosrun/roslaunch
.
Author: David, Henry, Qinru, Hao.