Perception Research

This repository is used as a playground to run perception experiments. Each of these are containerized through Docker. Current projects include

• Using transfer learning with the YOLOv5 model (by freezing backbone layers) to perform traffic sign detection
• Synthetic Data Generation for Traffic Signs using CARLA
• Multimodal Object Deteciton with BEVFusion

Pre-requisite: Before you run any of the commands below, make sure you are familiar with Docker. If not, we highly recommend going through this 2-hour video which teaches you the basics of Docker.

Getting Started

To get started, run the following script to setup some environment variables

./initialize.sh

Then, run the docker compose up command, followed by the name of the service you are interested in (if you don't specify, you will end up launching every single service...). You can find all the services we have under docker-compose.yml.

docker compose -p <userId> up <imageName> # ex: docker compose -p s36gong up yolov5

Then, to enter the terminal of the Docker container, open a new terminal and run

docker exec -it <ContainerID> /bin/bash

If you want to develop from the inside container itself, we recommend using VSCode built-in Docker container.

File Structure

• docker/ contains a set of custom Dockerfile to build our own containers
• src/ contains the source code for our investigation projects, usually accompanied their own separate README.md to explain how the project is being conducted.

Datasets

We have a few datasets downloaded already on our servers. The following are the paths to access the datasets:

General

• KITTI: /mnt/wato-drive/perception/KITTI
• nuScenes: /mnt/wato-drive/perception/nuscenes_CLEAN

The following are only accessible if you are on the trpro server. We downloaded the datasets specifically on the trpro machine to have faster write and read when processing the dataset:

• nuScenes: /mnt/scratch/nuscenes_CLEAN