-
Notifications
You must be signed in to change notification settings - Fork 70
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Add distributed version of the h.264 Endoscopy Tool Tracking app (#496)
* Add distributed version of the h.264 Endoscopy Tool Tracking app in C++ & Python Signed-off-by: Victor Chang <[email protected]> * Fix lint issues Signed-off-by: Victor Chang <[email protected]> * Address feedback Signed-off-by: Victor Chang <[email protected]> * Update extension paths Signed-off-by: Victor Chang <[email protected]> --------- Signed-off-by: Victor Chang <[email protected]>
- Loading branch information
Showing
22 changed files
with
1,620 additions
and
17 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
20 changes: 20 additions & 0 deletions
20
applications/h264/h264_endoscopy_tool_tracking_distributed/CMakeLists.txt
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,20 @@ | ||
| # SPDX-FileCopyrightText: Copyright (c) 2024 NVIDIA CORPORATION & AFFILIATES. All rights reserved. | ||
| # SPDX-License-Identifier: Apache-2.0 | ||
| # | ||
| # Licensed under the Apache License, Version 2.0 (the "License"); | ||
| # you may not use this file except in compliance with the License. | ||
| # You may obtain a copy of the License at | ||
| # | ||
| # http://www.apache.org/licenses/LICENSE-2.0 | ||
| # | ||
| # Unless required by applicable law or agreed to in writing, software | ||
| # distributed under the License is distributed on an "AS IS" BASIS, | ||
| # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| # See the License for the specific language governing permissions and | ||
| # limitations under the License. | ||
|
|
||
| cmake_minimum_required(VERSION 3.20) | ||
| project(h264_endoscopy_tool_tracking_distributed_apps LANGUAGES NONE) | ||
|
|
||
| add_subdirectory(cpp) | ||
| add_subdirectory(python) |
82 changes: 82 additions & 0 deletions
82
applications/h264/h264_endoscopy_tool_tracking_distributed/README.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,82 @@ | ||
| # Distributed H.264 Endoscopy Tool Tracking Application | ||
|
|
||
| This application is similar to the [H.264 Endoscopy Tool Tracking](../h264_endoscopy_tool_tracking/) application, but this distributed version divides the application into three fragments: | ||
|
|
||
| 1. Video Input: get video input from a pre-recorded video file. | ||
| 2. Inference: run the inference using LSTM and run the post-processing script. | ||
| 3. Visualization: display input video and inference results. | ||
|
|
||
|
|
||
| ## Requirements | ||
|
|
||
| This application is configured to use H.264 elementary stream from endoscopy sample data as input. | ||
|
|
||
| ### Data | ||
|
|
||
| [📦️ (NGC) Sample App Data for AI-based Endoscopy Tool Tracking](https://catalog.ngc.nvidia.com/orgs/nvidia/teams/clara-holoscan/resources/holoscan_endoscopy_sample_data) | ||
|
|
||
| The data is automatically downloaded when building the application. | ||
|
|
||
| ## Building and Running H.264 Endoscopy Tool Tracking Application | ||
|
|
||
| * Building and running the application from the top level Holohub directory: | ||
|
|
||
| ### C++ | ||
|
|
||
| ```bash | ||
| # Start the application with all three fragments | ||
| ./dev_container build_and_run h264_endoscopy_tool_tracking_distributed --docker_file applications/h264/Dockerfile --language cpp | ||
|
|
||
| # Use the following commands to run the same application three processes: | ||
| # Start the application with the video_in fragment | ||
| ./dev_container build_and_run h264_endoscopy_tool_tracking_distributed --docker_file applications/h264/Dockerfile --language cpp --run_args "--driver --worker --fragments video_in --address :10000 --worker-address :10001" | ||
| # Start the application with the inference fragment | ||
| ./dev_container build_and_run h264_endoscopy_tool_tracking_distributed --docker_file applications/h264/Dockerfile --language cpp --run_args "--worker --fragments inference --address :10000 --worker-address :10002" | ||
| # Start the application with the visualization fragment | ||
| ./dev_container build_and_run h264_endoscopy_tool_tracking_distributed --docker_file applications/h264/Dockerfile --language cpp --run_args "--worker --fragments viz --address :10000 --worker-address :10003" | ||
| ``` | ||
| --base_img gitlab-master.nvidia.com:5005/holoscan/holoscan-sdk/dev-x86_64:main | ||
| ### Python | ||
|
|
||
| ```bash | ||
| # Start the application with all three fragments | ||
| ./dev_container build_and_run h264_endoscopy_tool_tracking_distributed --docker_file applications/h264/Dockerfile --language python | ||
|
|
||
| # Use the following commands to run the same application three processes: | ||
| # Start the application with the video_in fragment | ||
| ./dev_container build_and_run h264_endoscopy_tool_tracking_distributed --docker_file applications/h264/Dockerfile --language python --run_args "--driver --worker --fragments video_in --address :10000 --worker-address :10001" | ||
| # Start the application with the inference fragment | ||
| ./dev_container build_and_run h264_endoscopy_tool_tracking_distributed --docker_file applications/h264/Dockerfile --language python --run_args "--worker --fragments inference --address :10000 --worker-address :10002" | ||
| # Start the application with the visualization fragment | ||
| ./dev_container build_and_run h264_endoscopy_tool_tracking_distributed --docker_file applications/h264/Dockerfile --language python --run_args "--worker --fragments viz --address :10000 --worker-address :10003" | ||
| ``` | ||
|
|
||
| Important: on aarch64, applications also need tegra folder mounted inside the container and | ||
| the `LD_LIBRARY_PATH` environment variable should be updated to include | ||
| tegra folder path. | ||
|
|
||
| Open and edit the [Dockerfile](../Dockerfile) and uncomment line 66: | ||
|
|
||
| ```bash | ||
| # Uncomment the following line for aarch64 support | ||
| ENV LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/lib/aarch64-linux-gnu/tegra/ | ||
| ``` | ||
|
|
||
|
|
||
| ## Dev Container | ||
|
|
||
| To start the VS Code Dev Container, run the following command from the root directory of Holohub: | ||
|
|
||
| ```bash | ||
| ./dev_container vscode h264 | ||
| ``` | ||
|
|
||
| ### VS Code Launch Profiles | ||
|
|
||
| #### C++ | ||
|
|
||
| Use the **(gdb) h264_endoscopy_tool_tracking_distributed/cpp (all fragments)** launch profile to run and debug the C++ application. | ||
|
|
||
| #### Python | ||
|
|
||
| Use the **(pythoncpp) h264_endoscopy_tool_tracking_distributed/python (all fragments)** launch profile to run and debug the Python application. |
59 changes: 59 additions & 0 deletions
59
applications/h264/h264_endoscopy_tool_tracking_distributed/cpp/CMakeLists.txt
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,59 @@ | ||
| # SPDX-FileCopyrightText: Copyright (c) 2024 NVIDIA CORPORATION & AFFILIATES. All rights reserved. | ||
| # SPDX-License-Identifier: Apache-2.0 | ||
| # | ||
| # Licensed under the Apache License, Version 2.0 (the "License"); | ||
| # you may not use this file except in compliance with the License. | ||
| # You may obtain a copy of the License at | ||
| # | ||
| # http://www.apache.org/licenses/LICENSE-2.0 | ||
| # | ||
| # Unless required by applicable law or agreed to in writing, software | ||
| # distributed under the License is distributed on an "AS IS" BASIS, | ||
| # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| # See the License for the specific language governing permissions and | ||
| # limitations under the License. | ||
|
|
||
| cmake_minimum_required(VERSION 3.20) | ||
| project(h264_endoscopy_tool_tracking_distributed CXX) | ||
|
|
||
| find_package(holoscan 2.5 REQUIRED CONFIG | ||
| PATHS "/opt/nvidia/holoscan" "/workspace/holoscan-sdk/install") | ||
|
|
||
| add_executable(h264_endoscopy_tool_tracking_distributed | ||
| main.cpp | ||
| ) | ||
|
|
||
| target_link_libraries(h264_endoscopy_tool_tracking_distributed | ||
| PRIVATE | ||
| holoscan::core | ||
| holoscan::ops::gxf_codelet | ||
| holoscan::ops::format_converter | ||
| holoscan::ops::holoviz | ||
| holoscan::ops::video_encoder | ||
| holoscan::ops::tensor_to_video_buffer | ||
| lstm_tensor_rt_inference | ||
| tool_tracking_postprocessor | ||
| ) | ||
|
|
||
| # Copy the config to the binary directory | ||
| add_custom_target(h264_endoscopy_tool_tracking_distributed_yaml | ||
| COMMAND ${CMAKE_COMMAND} -E copy_if_different "${CMAKE_CURRENT_SOURCE_DIR}/endoscopy_tool_tracking.yaml" ${CMAKE_CURRENT_BINARY_DIR} | ||
| DEPENDS "endoscopy_tool_tracking.yaml" | ||
| BYPRODUCTS "endoscopy_tool_tracking.yaml" | ||
| ) | ||
| add_dependencies(h264_endoscopy_tool_tracking_distributed h264_endoscopy_tool_tracking_distributed_yaml) | ||
|
|
||
| # Default to download datasets | ||
| option(HOLOHUB_DOWNLOAD_DATASETS "Download datasets" ON) | ||
|
|
||
| # Download the endoscopy sample data | ||
| if(HOLOHUB_DOWNLOAD_DATASETS) | ||
| include(holoscan_download_data) | ||
| holoscan_download_data(endoscopy | ||
| URL https://api.ngc.nvidia.com/v2/resources/nvidia/clara-holoscan/holoscan_endoscopy_sample_data/versions/20230222/zip | ||
| DOWNLOAD_NAME holoscan_endoscopy_sample_data_20230222.zip | ||
| URL_MD5 d54f84a562d29ed560a87d2607eba973 | ||
| DOWNLOAD_DIR ${HOLOHUB_DATA_DIR} | ||
| ) | ||
| add_dependencies(h264_endoscopy_tool_tracking_distributed endoscopy_data) | ||
| endif() |
Oops, something went wrong.