Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(yolox): updating autoware tensorrt yolox for type adaptation #9312

Open
wants to merge 7 commits into
base: main
Choose a base branch
from

Conversation

Owen-Liuyuxuan
Copy link
Contributor

@Owen-Liuyuxuan Owen-Liuyuxuan commented Nov 13, 2024

Description

Updating type adaptation for Autoware YOLOX.

Included changes:

  1. Adding additional type adaptation packages defining image container and compressed image container.
    • Please check the readme of the package for more information.
  2. Adopting the core YOLOX interface to work with type adaptation.
  3. Slightly refactoring YOLOX node,
    • Organize part of codes for reusability.
    • Adding options for behavior optimization
      • choosing between activating type adaptation or not.
      • choosing to use image transport (which will process the image on the CPU) or not.
      • choosing to output the debug/rois/image for not (which will move the image to CPU for openCV drawing).

Limitation:

  • I have only covered functions that will be used by the tensorrt_yolox_node and functions including "multiScaleFeedForward" are not fully adapted. But it should not be difficult considering the most tricky parts only lie in the function "preprocessGPU"
  • Type adaptation now only works with GPU preprocessing.
  • We need more considerations on how to work with LiDAR.

Related links

Parent Issue:

  • None for the moment.

Private Links:

How was this PR tested?

For this PR, I launch six Tensorrt YOLOX nodes in parallel and use my personal codes to publish six streams of normal images there.

This test simply turns on/off type adaptation on the launch file, which means that this shows how running type adaptation codes without pipeline optimization does not make performance worse.

Type adaptation ON vs Type adaption OFF:
image

Tests that include streamlining multiple nodes with type adaptation and getting optimized results are presented in the TIERIV internal link

Notes for reviewers

None.

Interface changes

ROS Parameter Changes

Additions and removals

Change type Parameter Name Type Default Value Description
Added is_using_image_transport bool true If true, will use image_transport, false -> will only publish raw images
Added type_adaptation_activated bool true If true, will use type_adaptation to transfer images in GPU.
Added is_publish_debug_rois_image_ bool true If true, will draw ROIs in the image and publish it for debugging

Effects on system behavior

None.

@github-actions github-actions bot added type:documentation Creating or refining documentation. (auto-assigned) component:perception Advanced sensor data processing and environment understanding. (auto-assigned) component:common Common packages from the autoware-common repository. (auto-assigned) tag:require-cuda-build-and-test labels Nov 13, 2024
Copy link

github-actions bot commented Nov 13, 2024

Thank you for contributing to the Autoware project!

🚧 If your pull request is in progress, switch it to draft mode.

Please ensure:

@Owen-Liuyuxuan Owen-Liuyuxuan changed the title feat(yolox): adopting autoware tensorrt yolox for type adaptation feat(yolox): updating autoware tensorrt yolox for type adaptation Nov 13, 2024
pre-commit-ci bot and others added 4 commits November 13, 2024 07:10
Signed-off-by: YuxuanLiuTier4Desktop <[email protected]>
…n/autoware.universe into feat/type_adaptation_yolox
Signed-off-by: YuxuanLiuTier4Desktop <[email protected]>
@ktro2828
Copy link
Contributor

@Owen-Liuyuxuan Thank you for PR!
First of all, as of this conversation, CUDA related packages have been moved under sensing directory.
Then, it looks better to move autoware_type_adapters under sensing or autoware_cuda_utils.

Signed-off-by: YuxuanLiuTier4Desktop <[email protected]>
@github-actions github-actions bot added component:sensing Data acquisition from sensors, drivers, preprocessing. (auto-assigned) and removed component:common Common packages from the autoware-common repository. (auto-assigned) labels Nov 15, 2024
@Owen-Liuyuxuan
Copy link
Contributor Author

@ktro2828
Thanks for the information. I shifted the entire adapters into the sensing directory.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
component:perception Advanced sensor data processing and environment understanding. (auto-assigned) component:sensing Data acquisition from sensors, drivers, preprocessing. (auto-assigned) tag:require-cuda-build-and-test type:documentation Creating or refining documentation. (auto-assigned)
Projects
Status: In Progress
Development

Successfully merging this pull request may close these issues.

2 participants