Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

OpenPCDet LiDar Object Detection Node #73

Open
wants to merge 78 commits into
base: main
Choose a base branch
from
Open

OpenPCDet LiDar Object Detection Node #73

wants to merge 78 commits into from

Conversation

danielrhuynh
Copy link
Contributor

@danielrhuynh danielrhuynh commented Feb 9, 2024 •
edited
Loading

Overview

Added a ros node for LiDar object detection node trained on nuscenes point clouds.
source repo: https://github.com/open-mmlab/OpenPCDet

demos:

raw_lidar.mov
new.annotations.mov

@danielrhuynh
base node for openpcdet
aa95f29
@danielrhuynh
copied dockerfile for openpcdet
ff4a955
@danielrhuynh
working package in ros
4455065
@danielrhuynh
ig we dont need these
1af45db
@danielrhuynh
added python package if I want to use that
47b7163
@danielrhuynh
adding ros node
a42c5d6
@danielrhuynh
made node and WIP on dockerfile
afddd09
@danielrhuynh
builds with cuda and I can run the setup script :)
9a072d9
@danielrhuynh
Adding OpenCV
00eec7b
@danielrhuynh
ROS node runs and makes a single inference on a .bin!
3fad1f7
@danielrhuynh
Foxglove integration and bounding boxes
cab5d52
@danielrhuynh
aligned lidar data with bounding boxes
9832d19
@danielrhuynh
changing to work with wato fork of openpcdet
c0aa3e8
@danielrhuynh
base node for openpcdet
424af34
@danielrhuynh
copied dockerfile for openpcdet
46441ba
@danielrhuynh
working package in ros
e966132
@danielrhuynh
ig we dont need these
257407e
@danielrhuynh
added python package if I want to use that
f1c47c0
@danielrhuynh
adding ros node
db7c638
@danielrhuynh
made node and WIP on dockerfile
cec4f6d
@danielrhuynh
builds with cuda and I can run the setup script :)
1a7eedf
@danielrhuynh
Adding OpenCV
0346da7
@danielrhuynh
ROS node runs and makes a single inference on a .bin!
3164865
@danielrhuynh
Foxglove integration and bounding boxes
6ac0f6d
@danielrhuynh
aligned lidar data with bounding boxes
58f0e19
@danielrhuynh
changing to work with wato fork of openpcdet
2480fd4
@danielrhuynh
Merge branch 'openpcdet-dan' of https://github.com/WATonomous/wato_mo…
128cb63 
…norepo into openpcdet-dan
@danielrhuynh danielrhuynh changed the title Openpcdet dan OpenPCDet LiDar Object Detection Node Feb 9, 2024
Edwardius
Edwardius requested changes Jun 11, 2024
View reviewed changes
Copy link
Collaborator

@Edwardius Edwardius left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesomeeeee work! Just some minor comments here and there. How big is the Lidar Image?

docker/perception/lidar_object_detection/lidar_object_detection.Dockerfile Outdated Show resolved Hide resolved
docker/perception/lidar_object_detection/lidar_object_detection.Dockerfile Outdated Show resolved Hide resolved
docker/perception/lidar_object_detection/lidar_object_detection.Dockerfile Outdated Show resolved Hide resolved
docker/perception/lidar_object_detection/lidar_object_detection.Dockerfile Outdated Show resolved Hide resolved
modules/docker-compose.perception.yaml Outdated Show resolved Hide resolved
...bject_detection/lidar_object_detection/lidar_object_detection/lidar_object_detection_node.py Outdated Show resolved Hide resolved
...bject_detection/lidar_object_detection/lidar_object_detection/lidar_object_detection_node.py Outdated Show resolved Hide resolved
...bject_detection/lidar_object_detection/lidar_object_detection/lidar_object_detection_node.py Outdated Show resolved Hide resolved
...bject_detection/lidar_object_detection/lidar_object_detection/lidar_object_detection_node.py Outdated Show resolved Hide resolved
...bject_detection/lidar_object_detection/lidar_object_detection/lidar_object_detection_node.py
detection.bbox.size.y = float(box[4])
detection.bbox.size.z = float(box[5])
detected_object = ObjectHypothesisWithPose()
detected_object.hypothesis.class_id = str(pred_dicts[0]["pred_labels"][idx])
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

these labels should be pulled out of the model and stored centrally @danielrhuynh in some sort of label server that publishes /vision_msgs/VisionInfo.msg

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not quite sure what this means lol

Copy link
Contributor Author

@danielrhuynh danielrhuynh Jun 11, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The pred labels? I think that comes from OpenPCDet so I'm a little hesitant to mess with those

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

class labels yeah, otherwise we don't know what the class_id actually refers too downstream

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

eg. how do we know 1 refers to a car?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gotcha, added

@danielrhuynh
Copy link
Contributor Author

danielrhuynh commented Jun 11, 2024 •
edited
Loading

Awesomeeeee work! Just some minor comments here and there. How big is the Lidar Image?

Thanks! Comments are for the most part addressed, the image is 900MB

@Edwardius
Copy link
Collaborator

Awesomeeeee work! Just some minor comments here and there. How big is the Lidar Image?

Thanks! Comments are for the most part addressed, the image is 900MB

oh! We shouldnt be skipping it's build then

@danielrhuynh danielrhuynh linked an issue Jun 14, 2024 that may be closed by this pull request
Address LiDAR Object Detection PR Comments #129
Open
@danielrhuynh
Copy link
Contributor Author

Awesomeeeee work! Just some minor comments here and there. How big is the Lidar Image?

Thanks! Comments are for the most part addressed, the image is 900MB

oh! We shouldnt be skipping it's build then

Screenshot 2024-06-27 at 9 11 55 PM
I lied big time, almost 19GB

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@leungjch leungjch Awaiting requested review from leungjch

@Gongsta Gongsta Awaiting requested review from Gongsta

@Edwardius Edwardius Awaiting requested review from Edwardius

Requested changes must be addressed to merge this pull request.

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Address LiDAR Object Detection PR Comments
4 participants
@danielrhuynh @Edwardius @Gongsta @dylanwang0