Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Buildfarmer log 2024/06/10 - 2024/06/17 #56

Closed
github-actions bot opened this issue Jun 10, 2024 · 17 comments
Closed

Buildfarmer log 2024/06/10 - 2024/06/17 #56

github-actions bot opened this issue Jun 10, 2024 · 17 comments
Labels
buildfarmer-log Log of issues in the buildfarm

Comments

@github-actions
Copy link
Contributor

Previous log #55

@github-actions github-actions bot added the buildfarmer-log Log of issues in the buildfarm label Jun 10, 2024
Copy link
Contributor Author

Jun 10

Build regressions
gz-fuel-tools10-debbuilder|364|2024-06-10 11:04:46.755000|Unknown Reason|create deb packages
Test regressions
Excluding 'performance, connext, fastrtps-dynamic' jobs
Issue #1 in nightly_linux_coverage
- Failing 100.0% in 15 out of 15 builds
	rosidl_typesupport_introspection_tests.MultiNestedMessageIntrospectionTest/0.CanWriteTypeErasedMessage
	projectroot.test_multi_nested_message_introspection
	  (also in: "nightly_win_rep" job)

Issue #2 in Rci__nightly-cyclonedds_ubuntu_noble_amd64
- Failing 100.0% in 13 out of 13 builds
	rclcpp.TestBusyWaiting/StaticSingleThreadedExecutor.test_spin_some
	rclcpp.TestBusyWaiting/SingleThreadedExecutor.test_spin_some
	rclcpp.TestBusyWaiting/SingleThreadedExecutor.test_spin_all
	rclcpp.TestBusyWaiting/MultiThreadedExecutor.test_spin_some
	rclcpp.TestBusyWaiting/MultiThreadedExecutor.test_spin_all
	  (also in: "Jci__nightly-cyclonedds_ubuntu_noble_amd64" job)

Issue #3 in Rci__benchmark_ubuntu_noble_amd64
- Failing 100.0% in 10 out of 10 builds
	projectroot.benchmark_string_conversions
	  (also in: "Jci__benchmark_ubuntu_noble_amd64" job)

Issue #4 in nightly_win_deb
- Failing 60.0% in 9 out of 15 builds
	sros2.test.sros2.commands.security.verbs.test_generate_policy_no_nodes.test_generate_policy_no_nodes

Issue #5 in gz_rendering-7-win
- Failing 100.0% in 5 out of 5 builds
	MeshTest.NormalMapWithoutTexCoord
	  (also in: "gz_rendering-main-win" job)
	  (also in: "gz_rendering-8-win" job)
	  (also in: "gz_rendering-7-win" job)

Issue #6 in nightly_win_rep
- Failing 45.45% in 5 out of 11 builds
	projectroot.test.rclcpp.test_executors (known issue in: https://github.com/ros2/rclcpp/issues/2502 - OPEN)
	  (also in: "nightly_win_rel" job) (known issue in: https://github.com/ros2/rclcpp/issues/2502 - OPEN)
	  (also in: "nightly_linux-aarch64_repeated" job) (known issue in: https://github.com/ros2/rclcpp/issues/2502 - OPEN)
	  (also in: "Rci__nightly-cyclonedds_ubuntu_noble_amd64" job) (known issue in: https://github.com/ros2/rclcpp/issues/2502 - OPEN)
	  (also in: "Jci__nightly-cyclonedds_ubuntu_noble_amd64" job) (known issue in: https://github.com/ros2/rclcpp/issues/2502 - OPEN)
	  (also in: "Ici__nightly-connext_ubuntu_jammy_amd64" job) (known issue in: https://github.com/ros2/rclcpp/issues/2502 - OPEN)

Issue #7 in nightly_linux_debug
- Failing 35.71% in 5 out of 14 builds
	demo_nodes_cpp.TestExecutablesTutorial.test_processes_output
	  (also in: "nightly_win_rep" job)
	  (also in: "nightly_win_rel" job)
	  (also in: "nightly_win_deb" job)
	  (also in: "nightly_linux_repeated" job)
	  (also in: "nightly_linux_debug" job)
	  (also in: "nightly_linux-rhel_repeated" job)
	  (also in: "nightly_linux-rhel_debug" job)

Issue #8 in gz_rendering-ci-main-noble-amd64
- Failing 100.0% in 4 out of 4 builds
	INTEGRATION_load_unload_ogre_gl3plus.test_ran

Issue #9 in nightly_linux_coverage
- Failing 26.67% in 4 out of 15 builds
	ros2topic.ros2topic.test.test_use_sim_time.test_use_sim_time (known issue in: https://github.com/ros2/ros2cli/issues/772 - CLOSED)

Issue #10 in nightly_linux_repeated
- Failing 26.67% in 4 out of 15 builds
	launch_testing_examples.launch_testing_examples.check_node_launch_test.launch_testing_examples.check_node_launch_test
	  (also in: "nightly_win_rep" job)
	  (also in: "nightly_linux_debug" job)
	  (also in: "nightly_linux-rhel_repeated" job)
	  (also in: "nightly_linux-aarch64_repeated" job)
	  (also in: "nightly_linux-aarch64_debug" job)
	  (also in: "Ici__nightly-connext_ubuntu_jammy_amd64" job)
	  (also in: "Hci__nightly-connext_ubuntu_jammy_amd64" job)

Issue #11 in nightly_win_rep
- Failing 36.36% in 4 out of 11 builds
	test_rclcpp.test_timeout_subscriber.timeout_subscriber

Issue #12 in gz_sensors-ci-main-noble-amd64
- Failing 60.0% in 3 out of 5 builds
	BoundingBoxCameraSensor/BoundingBoxCameraSensorTest.BoxesWithBuiltinSDF/ogre2
	<init>.BoundingBoxCameraSensor/BoundingBoxCameraSensorTest

Issue #13 in nightly_linux-rhel_repeated
- Failing 21.43% in 3 out of 14 builds
	ros2cli.ros2cli.test.test_strategy.test_with_daemon_spawn[1-3]
	  (also in: "nightly_linux_repeated" job)
	  (also in: "nightly_linux-aarch64_repeated" job)

Issue #14 in nightly_linux-rhel_repeated
- Failing 21.43% in 3 out of 14 builds
	projectroot.test_tf2_geometry_msgs

Issue #15 in nightly_linux_repeated
- Failing 20.0% in 3 out of 15 builds
	rclcpp.TestClockWakeup.multiple_threads_wait_on_one_clock
	projectroot.test.rclcpp.test_clock
	  (also in: "nightly_win_rep" job)
	  (also in: "nightly_linux-rhel_repeated" job)

Issue #16 in nightly_linux_repeated
- Failing 20.0% in 3 out of 15 builds
	launch_testing_examples.launch_testing_examples.check_multiple_nodes_launch_test.launch_testing_examples.check_multiple_nodes_launch_test (known issue in: https://github.com/ros2/examples/issues/340 - CLOSED)
	  (also in: "nightly_linux_debug" job) (known issue in: https://github.com/ros2/examples/issues/340 - CLOSED)

Issue #17 in nightly_win_rep
- Failing 27.27% in 3 out of 11 builds
	projectroot.test_service

Issue #18 in gz_sim-ci-ign-gazebo3-focal-amd64
- Failing 66.67% in 2 out of 3 builds
	INTEGRATION_gpu_lidar.test_ran

Issue #19 in gz_transport-ci-gz-transport13-homebrew-amd64
- Failing 40.0% in 2 out of 5 builds
	twoProcSrvCallWithoutOutput.ServiceList

Issue #20 in nightly_linux-rhel_repeated
- Failing 14.29% in 2 out of 14 builds
	projectroot.display_factory_test

Issue #21 in nightly_linux_humble_coverage
- Failing 14.29% in 2 out of 14 builds
	launch_testing_ros.test.examples.wait_for_topic_launch_test.wait_for_topic_launch_test (known issue in: https://github.com/ros2/launch_ros/issues/304 - CLOSED)
	  (also in: "nightly_linux_iron_coverage" job) (known issue in: https://github.com/ros2/launch_ros/issues/304 - CLOSED)
	  (also in: "Ici__nightly-debug_ubuntu_jammy_amd64" job) (known issue in: https://github.com/ros2/launch_ros/issues/304 - CLOSED)

Issue #22 in nightly_win_rep
- Failing 18.18% in 2 out of 11 builds
	rclpy.test_service.xunit.missing_result

Issue #23 in nightly_win_rep
- Failing 18.18% in 2 out of 11 builds
	projectroot.test_lifecycle_service_client (known issue in: https://github.com/ros2/rclcpp/issues/2472 - OPEN)
	  (also in: "nightly_win_rel" job) (known issue in: https://github.com/ros2/rclcpp/issues/2472 - OPEN)
	  (also in: "Rci__nightly-cyclonedds_ubuntu_noble_amd64" job) (known issue in: https://github.com/ros2/rclcpp/issues/2472 - OPEN)
	  (also in: "Jci__nightly-cyclonedds_ubuntu_noble_amd64" job) (known issue in: https://github.com/ros2/rclcpp/issues/2472 - OPEN)

Issue #24 in nightly_win_rep
- Failing 18.18% in 2 out of 11 builds
	projectroot.test_lifecycle_node
	  (also in: "nightly_linux-aarch64_repeated" job)

Issue #25 in nightly_win_rep
- Failing 18.18% in 2 out of 11 builds
	projectroot.test_client
	  (also in: "Rci__nightly-connext_ubuntu_noble_amd64" job)
	  (also in: "Jci__nightly-connext_ubuntu_noble_amd64" job)
	  (also in: "Hci__nightly-connext_ubuntu_jammy_amd64" job)

Issue #26 in nightly_win_rep
- Failing 18.18% in 2 out of 11 builds
	rclcpp_lifecycle.TestStateMachineInfo.transition_graph
	rclcpp_lifecycle.TestStateMachineInfo.available_transitions
	rclcpp_lifecycle.TestStateMachineInfo.available_states
	rclcpp_lifecycle.TestService.construction_and_destruction
	rclcpp_lifecycle.TestRegisterCustomCallbacks.custom_callbacks
	rclcpp_lifecycle.TestLifecycleServiceClient.construct_destruct
	rclcpp_lifecycle.TestDefaultStateMachine.wait_for_graph_change
	rclcpp_lifecycle.TestDefaultStateMachine.test_graph_topics
	rclcpp_lifecycle.TestDefaultStateMachine.test_graph_services_by_node
	rclcpp_lifecycle.TestDefaultStateMachine.test_graph_services
	rclcpp_lifecycle.TestDefaultStateMachine.test_getters
	rclcpp_lifecycle.TestDefaultStateMachine.test_callback_groups
	rclcpp_lifecycle.TestDefaultStateMachine.check_parameters
	rclcpp_lifecycle.TestClient.construction_with_free_function
	rclcpp_lifecycle.TestClient.construction_and_destruction
	rclcpp_lifecycle.TestCallbackExceptions.positive_on_error_with_code
	rclcpp_lifecycle.TestCallbackExceptions.positive_on_error
	rclcpp_lifecycle.TestCallbackExceptions.negative_on_error_with_code
	rclcpp_lifecycle.TestCallbackExceptions.negative_on_error
	rclcpp_lifecycle.PerTimerType/TestLifecyclePublisher.publish_managed_by_node/wall_timer (4-byte object <00-00 00-00>)                     
	projectroot.test_state_machine_info
	projectroot.test_register_custom_callbacks
	projectroot.test_lifecycle_publisher
	projectroot.test_callback_exceptions

Issue #27 in nightly_win_rep
- Failing 18.18% in 2 out of 11 builds
	rclcpp.TestInterfaceTraits.has_node_base_interface
	rclcpp.TestExecutors/SingleThreadedExecutor.detachOnDestruction
	projectroot.test.rclcpp.test_logger_service
	projectroot.test.rclcpp.test_interface_traits
	  (also in: "nightly_win_rel" job)

Issue #28 in nightly_win_rep
- Failing 18.18% in 2 out of 11 builds
	projectroot.gtest_timeout_subscriber__rmw_cyclonedds_cpp

Issue #29 in gz_transport-ign-transport8-win
- Failing 25.0% in 1 out of 4 builds
	playback.ReplayStep (known issue in: https://github.com/gazebosim/gz-transport/issues/195 - CLOSED)
	  (also in: "gz_transport-main-win" job) (known issue in: https://github.com/gazebosim/gz-transport/issues/195 - CLOSED)

Issue #30 in nightly_linux-aarch64_repeated
- Failing 6.25% in 1 out of 16 builds
	projectroot.test_record__rmw_cyclonedds_cpp
	  (also in: "nightly_linux-rhel_repeated" job)

Issue #31 in nightly_linux-aarch64_repeated
- Failing 6.25% in 1 out of 16 builds
	launch_testing_examples.launch_testing_examples.hello_world_launch_test.launch_testing_examples.hello_world_launch_test
	  (also in: "nightly_linux-rhel_repeated" job)

Issue #32 in nightly_linux_humble_coverage
- Failing 7.14% in 1 out of 14 builds
	ros2param.ros2param.test.test_verb_dump.test_verb_dump (known issue in: https://github.com/ros2/rmw_fastrtps/issues/531 - CLOSED | https://github.com/ros2/ros2cli/issues/861 - CLOSED)
	  (also in: "nightly_linux-rhel_repeated" job) (known issue in: https://github.com/ros2/rmw_fastrtps/issues/531 - CLOSED | https://github.com/ros2/ros2cli/issues/861 - CLOSED)

Issue #33 in nightly_win_rep
- Failing 9.09% in 1 out of 11 builds
	rclcpp.TestExecutors/StaticSingleThreadedExecutor.spin_some (known issue in: https://github.com/ros2/rclcpp/issues/2502 - OPEN)
	rclcpp.TestExecutors/SingleThreadedExecutor.spin_some_max_duration
	rclcpp.TestExecutors/SingleThreadedExecutor.spin_some (known issue in: https://github.com/ros2/rclcpp/issues/2502 - OPEN)
	rclcpp.TestExecutors/MultiThreadedExecutor.spin_some_max_duration
	rclcpp.TestExecutors/MultiThreadedExecutor.spin_some (known issue in: https://github.com/ros2/rclcpp/issues/2502 - OPEN)
	  (also in: "nightly_win_rep" job)
	  (also in: "Rci__nightly-cyclonedds_ubuntu_noble_amd64" job) (known issue in: https://github.com/ros2/rclcpp/issues/2502 - OPEN)
	  (also in: "Jci__nightly-cyclonedds_ubuntu_noble_amd64" job)

Issue #34 in nightly_win_rep
- Failing 9.09% in 1 out of 11 builds
	projectroot.test_action_client
	  (also in: "nightly_win_rel" job)

Issue #35 in nightly_win_rep
- Failing 9.09% in 1 out of 11 builds
	test_rclcpp.test_local_parameters.get_from_node_variant_type
	rclcpp_lifecycle.TestDefaultStateMachine.test_shutdown_on_dtor
	rclcpp_lifecycle.TestDefaultStateMachine.shutdown_from_each_primary_state
	rclcpp_lifecycle.TestDefaultStateMachine.lifecycle_subscriber
	rclcpp_lifecycle.TestDefaultStateMachine.good_mood
	rclcpp_lifecycle.TestDefaultStateMachine.declare_parameters
	rclcpp_lifecycle.TestDefaultStateMachine.bad_mood
	rclcpp.TestLoggerService.test_set_logging_level_with_partial_invalid_param
	rclcpp.TestLoggerService.test_set_logging_level_with_invalid_param
	rclcpp.TestLoggerService.test_set_and_get_multi_logging_level
	rclcpp.TestExecutors/StaticSingleThreadedExecutor.testSpinUntilSharedFutureComplete
	rclcpp.TestExecutors/StaticSingleThreadedExecutor.testSpinUntilFutureCompleteWithTimeout
	rclcpp.TestExecutors/StaticSingleThreadedExecutor.testSpinUntilFutureCompleteNoTimeout
	rclcpp.TestExecutors/StaticSingleThreadedExecutor.testSpinUntilFutureCompleteInterrupted
	rclcpp.TestExecutors/StaticSingleThreadedExecutor.testSpinUntilFutureComplete
	rclcpp.TestExecutors/StaticSingleThreadedExecutor.testSpinNodeUntilFutureCompleteNodePtr
	rclcpp.TestExecutors/StaticSingleThreadedExecutor.testSpinNodeUntilFutureCompleteNodeBasePtr
	rclcpp.TestExecutors/StaticSingleThreadedExecutor.testRaceConditionAddNode
	rclcpp.TestExecutors/StaticSingleThreadedExecutor.spin_some_max_duration
	rclcpp.TestExecutors/StaticSingleThreadedExecutor.spinWithTimer
	rclcpp.TestExecutors/StaticSingleThreadedExecutor.spinWhileAlreadySpinning
	rclcpp.TestExecutors/StaticSingleThreadedExecutor.spinAll
	rclcpp.TestExecutors/StaticSingleThreadedExecutor.emptyExecutor
	rclcpp.TestExecutors/StaticSingleThreadedExecutor.detachOnDestruction
	rclcpp.TestExecutors/StaticSingleThreadedExecutor.addTemporaryNode
	rclcpp.TestExecutors/StaticSingleThreadedExecutor.addNodeTwoExecutors
	rclcpp.TestExecutors/SingleThreadedExecutor.testSpinUntilSharedFutureComplete
	rclcpp.TestExecutors/SingleThreadedExecutor.testSpinUntilFutureCompleteWithTimeout
	rclcpp.TestExecutors/SingleThreadedExecutor.testSpinUntilFutureCompleteNoTimeout
	rclcpp.TestExecutors/SingleThreadedExecutor.testSpinUntilFutureCompleteInterrupted
	rclcpp.TestExecutors/SingleThreadedExecutor.testSpinUntilFutureComplete
	rclcpp.TestExecutors/SingleThreadedExecutor.testSpinNodeUntilFutureCompleteNodePtr
	rclcpp.TestExecutors/SingleThreadedExecutor.testSpinNodeUntilFutureCompleteNodeBasePtr
	rclcpp.TestExecutors/SingleThreadedExecutor.testRaceConditionAddNode
	rclcpp.TestExecutors/SingleThreadedExecutor.spinWithTimer
	rclcpp.TestExecutors/SingleThreadedExecutor.spinWhileAlreadySpinning
	rclcpp.TestExecutors/SingleThreadedExecutor.spinAll
	rclcpp.TestExecutors/SingleThreadedExecutor.emptyExecutor
	rclcpp.TestExecutors/SingleThreadedExecutor.addTemporaryNode
	rclcpp.TestExecutors/SingleThreadedExecutor.addNodeTwoExecutors
	rclcpp.TestExecutors/MultiThreadedExecutor.testSpinUntilSharedFutureComplete
	rclcpp.TestExecutors/MultiThreadedExecutor.testSpinUntilFutureCompleteWithTimeout
	rclcpp.TestExecutors/MultiThreadedExecutor.testSpinUntilFutureCompleteNoTimeout
	rclcpp.TestExecutors/MultiThreadedExecutor.testSpinUntilFutureCompleteInterrupted
	rclcpp.TestExecutors/MultiThreadedExecutor.testSpinUntilFutureComplete
	rclcpp.TestExecutors/MultiThreadedExecutor.testSpinNodeUntilFutureCompleteNodePtr
	rclcpp.TestExecutors/MultiThreadedExecutor.testSpinNodeUntilFutureCompleteNodeBasePtr
	rclcpp.TestExecutors/MultiThreadedExecutor.testRaceConditionAddNode
	rclcpp.TestExecutors/MultiThreadedExecutor.spinWithTimer
	rclcpp.TestExecutors/MultiThreadedExecutor.spinWhileAlreadySpinning
	rclcpp.TestExecutors/MultiThreadedExecutor.spinAll
	rclcpp.TestExecutors/MultiThreadedExecutor.emptyExecutor
	rclcpp.TestExecutors/MultiThreadedExecutor.detachOnDestruction
	rclcpp.TestExecutors/MultiThreadedExecutor.addTemporaryNode
	rclcpp.TestExecutors/MultiThreadedExecutor.addNodeTwoExecutors
	rclcpp.TestExecutors/EventsExecutor.emptyExecutor
	rclcpp.TestExecutors/EventsExecutor.detachOnDestruction
	rclcpp.TestExecutors/EventsExecutor.addTemporaryNode
	rcl_action.TestActionClientBaseFixture.test_action_client_init_fini_maybe_fail
	projectroot.gtest_local_parameters__rmw_fastrtps_cpp

@Blast545
Copy link
Contributor

Builds regressions on fuel-tools debbuilders, keeping track

Made our debbuilders start to fail on jammy and noble, see:
https://build.osrfoundation.org/job/gz-fuel-tools10-debbuilder/365/ (jammy)
https://build.osrfoundation.org/job/gz-fuel-tools10-debbuilder/364/ (noble)

The error is:

dh_install: warning: Cannot find (any matches for) "usr/share/gz/*/*.yaml" (tried in ., debian/tmp)

dh_install: warning: libgz-fuel-tools10 missing files: usr/share/gz/*/*.yaml
dh_install: error: missing files, aborting
make: *** [debian/rules:5: binary] Error 25

I think it will be fixed after: gazebo-release/gz-fuel-tools10-release#3

@Blast545
Copy link
Contributor

Reported on gazebo's Agenda for keeping track:

INTEGRATION_load_unload_ogre_gl3plus test fails on gz_rendering-ci-pr_any-noble-amd64 gazebosim/gz-rendering#1007
INTEGRATION_boundingbox_camera BoundingBoxCameraSensor test failing on Linux + main gazebosim/gz-sensors#440

@Blast545
Copy link
Contributor

Current known to be yellow jobs on Gazebo (and investigations)

(NEW opened)

Keeping track (might need extra help to solve) issues:

Copy link
Contributor Author

Jun 11

Build regressions
nightly_linux-rhel_repeated|1926|2024-06-11 04:00:01.734000|Early fail|No last section
Ici__nightly-performance_ubuntu_jammy_amd64|101|2024-06-11 03:18:32.908000|Jenkins channel closing down (Agent disconnected)|run tests
gz_sensors-8-win|53|2024-06-10 17:06:33.307000|Jenkins channel closing down (Agent disconnected)|vcpkg: install all dependencies
gz_launch-ign-launch5-win|53|2024-06-10 22:06:22.668000|Jenkins channel closing down (Agent disconnected)|vcpkg: install all dependencies
Test regressions
Excluding 'performance, connext, fastrtps-dynamic' jobs
Issue #1 in nightly_linux_coverage
- Failing 100.0% in 15 out of 15 builds
	rosidl_typesupport_introspection_tests.MultiNestedMessageIntrospectionTest/0.CanWriteTypeErasedMessage
	projectroot.test_multi_nested_message_introspection
	  (also in: "nightly_win_rep" job)

Issue #2 in Rci__nightly-cyclonedds_ubuntu_noble_amd64
- Failing 100.0% in 12 out of 12 builds
	rclcpp.TestExecutors/SingleThreadedExecutor.spin_some_max_duration
	rclcpp.TestExecutors/MultiThreadedExecutor.spin_some_max_duration
	  (also in: "nightly_win_rep" job)
	  (also in: "Jci__nightly-cyclonedds_ubuntu_noble_amd64" job)

Issue #3 in Rci__nightly-cyclonedds_ubuntu_noble_amd64
- Failing 100.0% in 12 out of 12 builds
	rclcpp.TestBusyWaiting/StaticSingleThreadedExecutor.test_spin_some
	rclcpp.TestBusyWaiting/SingleThreadedExecutor.test_spin_some
	rclcpp.TestBusyWaiting/SingleThreadedExecutor.test_spin_all
	rclcpp.TestBusyWaiting/MultiThreadedExecutor.test_spin_some
	rclcpp.TestBusyWaiting/MultiThreadedExecutor.test_spin_all
	  (also in: "Jci__nightly-cyclonedds_ubuntu_noble_amd64" job)

Issue #4 in Rci__benchmark_ubuntu_noble_amd64
- Failing 100.0% in 11 out of 11 builds
	projectroot.benchmark_string_conversions
	  (also in: "Jci__benchmark_ubuntu_noble_amd64" job)

Issue #5 in nightly_win_deb
- Failing 60.0% in 9 out of 15 builds
	sros2.test.sros2.commands.security.verbs.test_generate_policy_no_nodes.test_generate_policy_no_nodes

Issue #6 in nightly_win_deb
- Failing 60.0% in 9 out of 15 builds
	demo_nodes_cpp.TestExecutablesTutorial.test_processes_output
	  (also in: "nightly_win_rep" job)
	  (also in: "nightly_win_rel" job)
	  (also in: "nightly_win_deb" job)
	  (also in: "nightly_linux_repeated" job)
	  (also in: "nightly_linux_debug" job)
	  (also in: "nightly_linux-rhel_repeated" job)
	  (also in: "nightly_linux-rhel_debug" job)

Issue #7 in nightly_win_rep
- Failing 50.0% in 6 out of 12 builds
	projectroot.test.rclcpp.test_executors (known issue in: https://github.com/ros2/rclcpp/issues/2502 - OPEN)
	  (also in: "nightly_win_rel" job) (known issue in: https://github.com/ros2/rclcpp/issues/2502 - OPEN)
	  (also in: "nightly_linux-aarch64_repeated" job) (known issue in: https://github.com/ros2/rclcpp/issues/2502 - OPEN)
	  (also in: "Rci__nightly-cyclonedds_ubuntu_noble_amd64" job) (known issue in: https://github.com/ros2/rclcpp/issues/2502 - OPEN)
	  (also in: "Jci__nightly-cyclonedds_ubuntu_noble_amd64" job) (known issue in: https://github.com/ros2/rclcpp/issues/2502 - OPEN)
	  (also in: "Ici__nightly-connext_ubuntu_jammy_amd64" job) (known issue in: https://github.com/ros2/rclcpp/issues/2502 - OPEN)

Issue #8 in gz_rendering-ci-main-noble-amd64
- Failing 100.0% in 4 out of 4 builds
	INTEGRATION_load_unload_ogre_gl3plus.test_ran

Issue #9 in gz_rendering-7-win
- Failing 100.0% in 4 out of 4 builds
	MeshTest.NormalMapWithoutTexCoord
	  (also in: "gz_rendering-main-win" job)
	  (also in: "gz_rendering-8-win" job)
	  (also in: "gz_rendering-7-win" job)

Issue #10 in nightly_linux_coverage
- Failing 26.67% in 4 out of 15 builds
	ros2topic.ros2topic.test.test_use_sim_time.test_use_sim_time (known issue in: https://github.com/ros2/ros2cli/issues/772 - CLOSED)

Issue #11 in nightly_linux_repeated
- Failing 26.67% in 4 out of 15 builds
	test_rclcpp.TestTwoExecutablesAfterShutdown.test_client_scope_cpp

Issue #12 in nightly_linux_repeated
- Failing 26.67% in 4 out of 15 builds
	rclcpp.TestClockWakeup.multiple_threads_wait_on_one_clock
	projectroot.test.rclcpp.test_clock
	  (also in: "nightly_win_rep" job)
	  (also in: "nightly_linux-rhel_repeated" job)

Issue #13 in nightly_win_deb
- Failing 26.67% in 4 out of 15 builds
	test_rclcpp.test_multithreaded.multi_access_publisher
	  (also in: "nightly_win_rep" job)
	  (also in: "nightly_win_deb" job)

Issue #14 in gz_sensors-ci-main-noble-amd64
- Failing 60.0% in 3 out of 5 builds
	BoundingBoxCameraSensor/BoundingBoxCameraSensorTest.BoxesWithBuiltinSDF/ogre2
	<init>.BoundingBoxCameraSensor/BoundingBoxCameraSensorTest

Issue #15 in nightly_linux_humble_coverage
- Failing 20.0% in 3 out of 15 builds
	launch_testing_ros.test.examples.wait_for_topic_launch_test.wait_for_topic_launch_test (known issue in: https://github.com/ros2/launch_ros/issues/304 - CLOSED)
	  (also in: "nightly_linux_iron_coverage" job) (known issue in: https://github.com/ros2/launch_ros/issues/304 - CLOSED)
	  (also in: "Ici__nightly-debug_ubuntu_jammy_amd64" job) (known issue in: https://github.com/ros2/launch_ros/issues/304 - CLOSED)

Issue #16 in nightly_linux_repeated
- Failing 20.0% in 3 out of 15 builds
	projectroot.test_client_scope_cpp__rmw_cyclonedds_cpp

Issue #17 in nightly_win_rep
- Failing 25.0% in 3 out of 12 builds
	rclcpp.test_executor.gtest.missing_result
	projectroot.test.rclcpp.test_executor (known issue in: https://github.com/ros2/rclcpp/issues/2502 - OPEN)
	  (also in: "nightly_win_rel" job)

Issue #18 in nightly_win_rep
- Failing 16.67% in 2 out of 12 builds
	test_security.TestSecurePublisherSubscriberAfterShutdown.test_processes_finished_gracefully
	  (also in: "nightly_linux_release" job)
	  (also in: "nightly_linux_humble_coverage" job)
	  (also in: "nightly_linux-rhel_release" job)

Issue #19 in nightly_win_rep
- Failing 16.67% in 2 out of 12 builds
	rclcpp.TestExecutors/StaticSingleThreadedExecutor.addTemporaryNode

Issue #20 in gz_sim-ci-ign-gazebo3-focal-amd64
- Failing 25.0% in 1 out of 4 builds
	CameraSensorBackgroundFixture.RedBackground

Issue #21 in nightly_win_rep
- Failing 8.33% in 1 out of 12 builds
	projectroot.test_secure_publisher_subscriber__Empty__rmw_fastrtps_cpp__non_secure_comm_1
	projectroot.point_cloud_transport_publisher
	point_cloud_transport.TestPublisher.publisher
	point_cloud_transport.TestPublisher.point_cloud_transport_publisher

@Crola1702
Copy link
Contributor

Humble fastrtps failure | Build regression | Keeping track

Reference build: https://build.ros2.org/view/Hci/job/Hci__nightly-fastrtps_ubuntu_jammy_amd64/249/

Log output:

ERROR: Step ‘Archive the artifacts’ failed: no workspace for Hci__nightly-fastrtps_ubuntu_jammy_amd64 #249
ERROR: Step ‘Publish xUnit test result report’ failed: no workspace for Hci__nightly-fastrtps_ubuntu_jammy_amd64 #249
Finished: FAILURE 

It seems it failed just at the end. There are no package nor test failures. The stderrors are the same as the passing build before.

Maybe de workspace was cleaned earlier and made the build fail.

@Crola1702
Copy link
Contributor

gz-sim windows | Test regression | Investigating 🕵️

Reference build: https://build.osrfoundation.org/job/gz_sim-8-win/78/

Test regression: Segfault from INTEGRATION_multiple_servers.test_ran in test: ServerRepeat/MultipleServers.TwoServersNonBlocking/0

Log output:

181: [ RUN      ] ServerRepeat/MultipleServers.TwoServersNonBlocking/0
...
181: �[m�[32m[Msg] �[m�[32mPublishing scene information on [�[m�[32mPublishing scene information on [�[m�[32m/world/default/scene/info�[m�[32m[Msg] �[m�[32m]�[m�[32m
181: �[m�[32m
181: Publishing entity deletions on [�[m�[32m/world/default

181/248 Test #181: INTEGRATION_multiple_servers ...........................***Exception: SegFault  2.70 sec

Test flakiness:

job_name last_fail first_fail build_count failure_count failure_percentage
gz_sim-main-win 2024-06-02 2024-06-02 11 1 9.09
gz_sim-8-win 2024-05-31 2024-05-31 7 1 14.29
gz_sim-ign-gazebo6-win 2024-05-28 2024-05-28 9 1 11.11
gz_sim-7-win 2024-05-27 2024-05-27 9 1 11.11

It rarely happens, but in all win distributions

Copy link
Contributor Author

Jun 12

Build regressions

Test regressions
Excluding 'performance, connext, fastrtps-dynamic' jobs
Issue #1 in nightly_linux_coverage
- Failing 100.0% in 14 out of 14 builds
	rosidl_typesupport_introspection_tests.MultiNestedMessageIntrospectionTest/0.CanWriteTypeErasedMessage
	projectroot.test_multi_nested_message_introspection
	  (also in: "nightly_win_rep" job)

Issue #2 in Rci__nightly-cyclonedds_ubuntu_noble_amd64
- Failing 100.0% in 12 out of 12 builds
	rclcpp.TestExecutors/SingleThreadedExecutor.spin_some_max_duration
	rclcpp.TestExecutors/MultiThreadedExecutor.spin_some_max_duration
	  (also in: "nightly_win_rep" job)
	  (also in: "Jci__nightly-cyclonedds_ubuntu_noble_amd64" job)

Issue #3 in Rci__nightly-cyclonedds_ubuntu_noble_amd64
- Failing 100.0% in 12 out of 12 builds
	rclcpp.TestBusyWaiting/StaticSingleThreadedExecutor.test_spin_some
	rclcpp.TestBusyWaiting/SingleThreadedExecutor.test_spin_some
	rclcpp.TestBusyWaiting/SingleThreadedExecutor.test_spin_all
	rclcpp.TestBusyWaiting/MultiThreadedExecutor.test_spin_some
	rclcpp.TestBusyWaiting/MultiThreadedExecutor.test_spin_all
	  (also in: "Jci__nightly-cyclonedds_ubuntu_noble_amd64" job)

Issue #4 in Rci__benchmark_ubuntu_noble_amd64
- Failing 100.0% in 11 out of 11 builds
	projectroot.benchmark_string_conversions
	  (also in: "Jci__benchmark_ubuntu_noble_amd64" job)

Issue #5 in nightly_win_deb
- Failing 57.14% in 8 out of 14 builds
	sros2.test.sros2.commands.security.verbs.test_generate_policy_no_nodes.test_generate_policy_no_nodes

Issue #6 in nightly_win_deb
- Failing 57.14% in 8 out of 14 builds
	demo_nodes_cpp.TestExecutablesTutorial.test_processes_output
	  (also in: "nightly_win_rep" job)
	  (also in: "nightly_win_rel" job)
	  (also in: "nightly_win_deb" job)
	  (also in: "nightly_linux_repeated" job)
	  (also in: "nightly_linux_debug" job)
	  (also in: "nightly_linux-rhel_repeated" job)
	  (also in: "nightly_linux-rhel_debug" job)

Issue #7 in nightly_win_rep
- Failing 54.55% in 6 out of 11 builds
	projectroot.test.rclcpp.test_executors (known issue in: https://github.com/ros2/rclcpp/issues/2502 - OPEN)
	  (also in: "nightly_win_rel" job) (known issue in: https://github.com/ros2/rclcpp/issues/2502 - OPEN)
	  (also in: "nightly_linux-aarch64_repeated" job) (known issue in: https://github.com/ros2/rclcpp/issues/2502 - OPEN)
	  (also in: "Rci__nightly-cyclonedds_ubuntu_noble_amd64" job) (known issue in: https://github.com/ros2/rclcpp/issues/2502 - OPEN)
	  (also in: "Jci__nightly-cyclonedds_ubuntu_noble_amd64" job) (known issue in: https://github.com/ros2/rclcpp/issues/2502 - OPEN)
	  (also in: "Ici__nightly-connext_ubuntu_jammy_amd64" job) (known issue in: https://github.com/ros2/rclcpp/issues/2502 - OPEN)

Issue #8 in gz_rendering-7-win
- Failing 100.0% in 4 out of 4 builds
	MeshTest.NormalMapWithoutTexCoord
	  (also in: "gz_rendering-main-win" job)
	  (also in: "gz_rendering-8-win" job)
	  (also in: "gz_rendering-7-win" job)

Issue #9 in nightly_linux_repeated
- Failing 26.67% in 4 out of 15 builds
	test_rclcpp.TestTwoExecutablesAfterShutdown.test_client_scope_cpp

Issue #10 in nightly_linux_repeated
- Failing 26.67% in 4 out of 15 builds
	rclcpp.TestClockWakeup.multiple_threads_wait_on_one_clock
	projectroot.test.rclcpp.test_clock
	  (also in: "nightly_win_rep" job)
	  (also in: "nightly_linux-rhel_repeated" job)

Issue #11 in nightly_win_deb
- Failing 28.57% in 4 out of 14 builds
	test_rclcpp.test_multithreaded.multi_access_publisher
	  (also in: "nightly_win_rep" job)
	  (also in: "nightly_win_deb" job)

Issue #12 in gz_rendering-ci-main-noble-amd64
- Failing 100.0% in 3 out of 3 builds
	INTEGRATION_load_unload_ogre_gl3plus.test_ran

Issue #13 in nightly_linux_coverage
- Failing 21.43% in 3 out of 14 builds
	ros2topic.ros2topic.test.test_use_sim_time.test_use_sim_time (known issue in: https://github.com/ros2/ros2cli/issues/772 - CLOSED)

Issue #14 in nightly_linux_humble_coverage
- Failing 20.0% in 3 out of 15 builds
	launch_testing_ros.test.examples.wait_for_topic_launch_test.wait_for_topic_launch_test (known issue in: https://github.com/ros2/launch_ros/issues/304 - CLOSED)
	  (also in: "nightly_linux_iron_coverage" job) (known issue in: https://github.com/ros2/launch_ros/issues/304 - CLOSED)
	  (also in: "Ici__nightly-debug_ubuntu_jammy_amd64" job) (known issue in: https://github.com/ros2/launch_ros/issues/304 - CLOSED)

Issue #15 in nightly_linux_repeated
- Failing 20.0% in 3 out of 15 builds
	projectroot.test_client_scope_cpp__rmw_cyclonedds_cpp

Issue #16 in nightly_win_rep
- Failing 27.27% in 3 out of 11 builds
	rclcpp.test_executor.gtest.missing_result
	projectroot.test.rclcpp.test_executor (known issue in: https://github.com/ros2/rclcpp/issues/2502 - OPEN)
	  (also in: "nightly_win_rel" job)

Issue #17 in nightly_win_rep
- Failing 18.18% in 2 out of 11 builds
	test_security.TestSecurePublisherSubscriberAfterShutdown.test_processes_finished_gracefully
	  (also in: "nightly_linux_release" job)
	  (also in: "nightly_linux_humble_coverage" job)

Issue #18 in nightly_win_rep
- Failing 18.18% in 2 out of 11 builds
	rclcpp.TestExecutors/StaticSingleThreadedExecutor.addTemporaryNode

Issue #19 in gz_sim-ci-ign-gazebo3-focal-amd64
- Failing 25.0% in 1 out of 4 builds
	CameraSensorBackgroundFixture.RedBackground

Issue #20 in nightly_win_rep
- Failing 9.09% in 1 out of 11 builds
	projectroot.test_secure_publisher_subscriber__Empty__rmw_fastrtps_cpp__non_secure_comm_1
	projectroot.point_cloud_transport_publisher
	point_cloud_transport.TestPublisher.publisher
	point_cloud_transport.TestPublisher.point_cloud_transport_publisher

@Crola1702
Copy link
Contributor

Crola1702 commented Jun 12, 2024

Ros2 issues report

Flaky ros2/demos#693:

Regressions:

  • demo_nodes_cpp.TestExecutablesTutorial.test_processes_output
  • projectroot.test_tutorial_parameter_events_async__rmw_connextdds
  • projectroot.test_tutorial_parameter_events__rmw_connextdds

Jobs:

  • Windows debug, release, repeated (consistent in windows debug)
  • Linux debug,
  • Rhel repeated, debug

@Crola1702
Copy link
Contributor

Linux repeated | Test regressions | Investigating 🕵️

Reference build: https://ci.ros2.org/view/nightly/job/nightly_linux_repeated/3483/

Test regressions:

Log output:
FAIL: test_client_scope_cpp (test_rclcpp.TestTwoExecutablesAfterShutdown.test_client_scope_cpp)
Test that both executables finished cleanly.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/jenkins-agent/workspace/nightly_linux_repeated/ws/build/test_rclcpp/test_client_scope_cpp__rmw_cyclonedds_cpp_Release.py", line 75, in test_client_scope_cpp
    launch_testing.asserts.assertExitCodes(
  File "/home/jenkins-agent/workspace/nightly_linux_repeated/ws/install/launch_testing/lib/python3.12/site-packages/launch_testing/asserts/assert_exit_codes.py", line 62, in assertExitCodes
    assert info.returncode in allowable_exit_codes, 'Proc {} exited with code {}'.format(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: Proc test_client_scope_server_cpp-1 exited with code -6
Log output:
2: Test command: /usr/bin/python3 "-u" "/home/jenkins-agent/workspace/nightly_linux_repeated/ws/install/ament_cmake_test/share/ament_cmake_test/cmake/run_test.py" "/home/jenkins-agent/worksp
2: Working Directory: /home/jenkins-agent/workspace/nightly_linux_repeated/ws/build/test_security^M
2: Test timeout computed to be: 10^M
2: -- run_test.py: extra environment variables:^M
2:  - PATH=":/home/jenkins-agent/workspace/nightly_linux_repeated/ws/install/ros2cli/bin:/home/rosbuild/rti_connext_dds-6.0.1/lib/x64Linux4gcc7.3.0:/home/rosbuild/rti_connext_dds-6.0.1/bin:/
2:  - RCL_ASSERT_RMW_ID_MATCHES=rmw_connextdds^M
2:  - RMW_IMPLEMENTATION=rmw_connextdds^M
2:  - ROS_SECURITY_KEYSTORE=/home/jenkins-agent/workspace/nightly_linux_repeated/ws/build/test_security/test/test_security_files^M
2: -- run_test.py: extra environment variables to append:^M
2:  - LD_LIBRARY_PATH+=/home/jenkins-agent/workspace/nightly_linux_repeated/ws/build/test_security:/home/rosbuild/rti_connext_dds-6.0.1/resource/app/lib/x64Linux2.6gcc4.4.5^M
2: -- run_test.py: invoking following command in '/home/jenkins-agent/workspace/nightly_linux_repeated/ws/build/test_security':^M
2:  - /home/jenkins-agent/workspace/nightly_linux_repeated/ws/build/test_security/test_security_nodes_c__rmw_connextdds --gtest_output=xml:/home/jenkins-agent/workspace/nightly_linux_repeate
    Test  #2: test_security_nodes_c__rmw_connextdds ...............................................................***Timeout  13.87 sec^M
Log output:
[ RUN      ] TestClockWakeup.multiple_threads_wait_on_one_clock
/home/jenkins-agent/workspace/nightly_linux_repeated/ws/src/ros2/rclcpp/rclcpp/test/rclcpp/test_clock.cpp:221: Failure
Value of: finished
  Actual: false
Expected: true

/home/jenkins-agent/workspace/nightly_linux_repeated/ws/src/ros2/rclcpp/rclcpp/test/rclcpp/test_clock.cpp:228: Failure
Expected: (cur_time) < (start_time + std::chrono::seconds(1)), actual: 8-byte object <33-7B CD-32 85-27 00-00> vs 8-byte object <B2-29 6D-32 85-27 00-00>

[  FAILED  ] TestClockWakeup.multiple_threads_wait_on_one_clock (1514 ms)

@Crola1702
Copy link
Contributor

Crola1702 commented Jun 12, 2024

Copy link
Contributor Author

Jun 13

Build regressions
packaging_linux-aarch64|2820|2024-06-13 04:00:01.944000|Failed to fetch some archives|Build Dockerfile
packaging_windows|3273|2024-06-13 04:00:01.964000|Early fail|No last section
gz-sim9-debbuilder|367|2024-06-12 14:27:11.100000|Failed to fetch some archives|install build dependencies from debian/control
gz_common-ci-main-homebrew-amd64|55|2024-06-12 19:06:36.697000|Network error: Failed to clone github repo|clean up /usr/local
gz_common-main-win|55|2024-06-12 22:06:39.451000|Jenkins channel closing down (Agent disconnected)|vcpkg: install all dependencies
gz_launch-main-win|51|2024-06-12 16:06:39.825000|Jenkins channel closing down (Agent disconnected)|colcon compilation without test for dependencies of gz-launch8
Test regressions
Excluding 'performance, connext, fastrtps-dynamic' jobs
Issue #1 in nightly_linux_coverage
- Failing 100.0% in 15 out of 15 builds
	rosidl_typesupport_introspection_tests.MultiNestedMessageIntrospectionTest/0.CanWriteTypeErasedMessage
	projectroot.test_multi_nested_message_introspection
	  (also in: "nightly_win_rep" job)
	  (also in: "nightly_linux_coverage" job)

Issue #2 in Rci__benchmark_ubuntu_noble_amd64
- Failing 100.0% in 10 out of 10 builds
	projectroot.benchmark_string_conversions
	  (also in: "Jci__benchmark_ubuntu_noble_amd64" job)

Issue #3 in nightly_win_deb
- Failing 53.33% in 8 out of 15 builds
	sros2.test.sros2.commands.security.verbs.test_generate_policy_no_nodes.test_generate_policy_no_nodes

Issue #4 in nightly_linux_repeated
- Failing 37.5% in 6 out of 16 builds
	rclcpp.TestClockWakeup.multiple_threads_wait_on_one_clock
	projectroot.test.rclcpp.test_clock
	  (also in: "nightly_win_rep" job)
	  (also in: "nightly_linux_repeated" job)
	  (also in: "nightly_linux-rhel_repeated" job)

Issue #5 in Jci__nightly-cyclonedds_ubuntu_noble_amd64
- Failing 100.0% in 5 out of 5 builds
	projectroot.test_lifecycle_service_client (known issue in: https://github.com/ros2/rclcpp/issues/2472 - OPEN)
	  (also in: "nightly_win_rep" job) (known issue in: https://github.com/ros2/rclcpp/issues/2472 - OPEN)
	  (also in: "nightly_win_rel" job) (known issue in: https://github.com/ros2/rclcpp/issues/2472 - OPEN)
	  (also in: "Rci__nightly-cyclonedds_ubuntu_noble_amd64" job) (known issue in: https://github.com/ros2/rclcpp/issues/2472 - OPEN)

Issue #6 in Jci__nightly-cyclonedds_ubuntu_noble_amd64
- Failing 100.0% in 5 out of 5 builds
	projectroot.test.rclcpp.test_executors (known issue in: https://github.com/ros2/rclcpp/issues/2502 - OPEN)
	  (also in: "nightly_win_rep" job) (known issue in: https://github.com/ros2/rclcpp/issues/2502 - OPEN)
	  (also in: "nightly_win_rel" job) (known issue in: https://github.com/ros2/rclcpp/issues/2502 - OPEN)
	  (also in: "Rci__nightly-cyclonedds_ubuntu_noble_amd64" job) (known issue in: https://github.com/ros2/rclcpp/issues/2502 - OPEN)
	  (also in: "Ici__nightly-connext_ubuntu_jammy_amd64" job) (known issue in: https://github.com/ros2/rclcpp/issues/2502 - OPEN)

Issue #7 in Jci__nightly-cyclonedds_ubuntu_noble_amd64
- Failing 100.0% in 5 out of 5 builds
	rclcpp.TestExecutors/StaticSingleThreadedExecutor.spin_some (known issue in: https://github.com/ros2/rclcpp/issues/2502 - OPEN)
	rclcpp.TestExecutors/SingleThreadedExecutor.spin_some_max_duration
	rclcpp.TestExecutors/SingleThreadedExecutor.spin_some (known issue in: https://github.com/ros2/rclcpp/issues/2502 - OPEN)
	rclcpp.TestExecutors/MultiThreadedExecutor.spin_some_max_duration
	rclcpp.TestExecutors/MultiThreadedExecutor.spin_some (known issue in: https://github.com/ros2/rclcpp/issues/2502 - OPEN)
	  (also in: "nightly_win_rep" job) (known issue in: https://github.com/ros2/rclcpp/issues/2502 - OPEN)
	  (also in: "Rci__nightly-cyclonedds_ubuntu_noble_amd64" job) (known issue in: https://github.com/ros2/rclcpp/issues/2502 - OPEN)
	  (also in: "Jci__nightly-cyclonedds_ubuntu_noble_amd64" job)

Issue #8 in Jci__nightly-cyclonedds_ubuntu_noble_amd64
- Failing 100.0% in 5 out of 5 builds
	rclpy.rclpy.test.test_guard_condition.TestGuardCondition.test_trigger (known issue in: https://github.com/ros2/rclpy/issues/1266 - OPEN)
	rclpy.rclpy.test.test_guard_condition.TestGuardCondition.test_double_trigger (known issue in: https://github.com/ros2/rclpy/issues/1266 - OPEN)
	rclcpp_lifecycle.TestLifecycleServiceClientRCLErrors.call_services_rcl_errors (known issue in: https://github.com/ros2/rclcpp/issues/2472 - OPEN)
	rclcpp.TestBusyWaiting/StaticSingleThreadedExecutor.test_spin_some
	rclcpp.TestBusyWaiting/SingleThreadedExecutor.test_spin_some
	rclcpp.TestBusyWaiting/SingleThreadedExecutor.test_spin_all
	rclcpp.TestBusyWaiting/MultiThreadedExecutor.test_spin_some
	rclcpp.TestBusyWaiting/MultiThreadedExecutor.test_spin_all
	projectroot.test_guard_condition (known issue in: https://github.com/ros2/rclpy/issues/1266 - OPEN)
	  (also in: "Rci__nightly-cyclonedds_ubuntu_noble_amd64" job) (known issue in: https://github.com/ros2/rclpy/issues/1266 - OPEN)
	  (also in: "Jci__nightly-cyclonedds_ubuntu_noble_amd64" job)

Issue #9 in gz_rendering-ci-main-noble-amd64
- Failing 100.0% in 5 out of 5 builds
	INTEGRATION_load_unload_ogre_gl3plus.test_ran

Issue #10 in nightly_win_deb
- Failing 33.33% in 5 out of 15 builds
	test_rclcpp.test_multithreaded.multi_access_publisher
	  (also in: "nightly_win_rep" job)

Issue #11 in nightly_linux-rhel_debug
- Failing 31.25% in 5 out of 16 builds
	demo_nodes_cpp.TestExecutablesTutorial.test_processes_output
	  (also in: "nightly_win_rep" job)
	  (also in: "nightly_win_rel" job)
	  (also in: "nightly_win_deb" job)
	  (also in: "nightly_linux_repeated" job)
	  (also in: "nightly_linux_debug" job)
	  (also in: "nightly_linux-rhel_repeated" job)
	  (also in: "nightly_linux-rhel_debug" job)

Issue #12 in gz_sensors-ci-main-noble-amd64
- Failing 80.0% in 4 out of 5 builds
	BoundingBoxCameraSensor/BoundingBoxCameraSensorTest.BoxesWithBuiltinSDF/ogre2
	<init>.BoundingBoxCameraSensor/BoundingBoxCameraSensorTest

Issue #13 in nightly_win_rep
- Failing 30.77% in 4 out of 13 builds
	rclpy.test_subscription.xunit.missing_result
	  (also in: "nightly_win_rel" job)

Issue #14 in nightly_win_rep
- Failing 30.77% in 4 out of 13 builds
	projectroot.test_subscription
	  (also in: "nightly_win_rel" job)
	  (also in: "Rci__nightly-fastrtps-dynamic_ubuntu_noble_amd64" job)
	  (also in: "Jci__nightly-fastrtps-dynamic_ubuntu_noble_amd64" job)
	  (also in: "Ici__nightly-fastrtps-dynamic_ubuntu_jammy_amd64" job)
	  (also in: "Ici__nightly-connext_ubuntu_jammy_amd64" job)

Issue #15 in nightly_win_rep
- Failing 30.77% in 4 out of 13 builds
	projectroot.test_publisher_subscriber__rclcpp__rmw_fastrtps_cpp

Issue #16 in gz_rendering-7-win
- Failing 100.0% in 3 out of 3 builds
	MeshTest.NormalMapWithoutTexCoord
	  (also in: "gz_rendering-main-win" job)
	  (also in: "gz_rendering-8-win" job)
	  (also in: "gz_rendering-7-win" job)

Issue #17 in nightly_win_rep
- Failing 23.08% in 3 out of 13 builds
	rclpy.test_waitable.xunit.missing_result
	projectroot.test_waitable
	  (also in: "nightly_win_rel" job)
	  (also in: "nightly_linux-aarch64_repeated" job)

Issue #18 in nightly_win_rep
- Failing 23.08% in 3 out of 13 builds
	projectroot.test_timer
	  (also in: "nightly_win_rel" job)
	  (also in: "nightly_linux_iron_coverage" job)

Issue #19 in nightly_win_rep
- Failing 23.08% in 3 out of 13 builds
	rclpy.test_wait_for_message.xunit.missing_result
	rclpy.test_timer.xunit.missing_result
	rclpy.test_time_source.xunit.missing_result
	projectroot.test_wait_for_message
	projectroot.test_time_source
	  (also in: "nightly_win_rel" job)

Issue #20 in nightly_win_rep
- Failing 23.08% in 3 out of 13 builds
	projectroot.test_service

Issue #21 in nightly_win_rep
- Failing 23.08% in 3 out of 13 builds
	projectroot.test.rclcpp.test_time (known issue in: https://github.com/ros2/rclcpp/issues/2436 - CLOSED)
	  (also in: "nightly_win_rel" job) (known issue in: https://github.com/ros2/rclcpp/issues/2436 - CLOSED)

Issue #22 in nightly_linux-rhel_repeated
- Failing 12.5% in 2 out of 16 builds
	test_rclcpp.TestNNodes.test_10_nodes
	  (also in: "nightly_win_rep" job)
	  (also in: "nightly_linux_repeated" job)

Issue #23 in nightly_linux-rhel_repeated
- Failing 12.5% in 2 out of 16 builds
	test_rclcpp.TestNNodesAfterShutdown.test_10_nodes
	projectroot.test_n_nodes__rmw_fastrtps_cpp
	  (also in: "nightly_win_rep" job)
	  (also in: "nightly_linux_repeated" job)

Issue #24 in nightly_linux_debug
- Failing 12.5% in 2 out of 16 builds
	launch_testing_examples.launch_testing_examples.check_multiple_nodes_launch_test.launch_testing_examples.check_multiple_nodes_launch_test (known issue in: https://github.com/ros2/examples/issues/340 - CLOSED)
	  (also in: "nightly_linux_repeated" job) (known issue in: https://github.com/ros2/examples/issues/340 - CLOSED)

Issue #25 in nightly_win_rep
- Failing 15.38% in 2 out of 13 builds
	test_communication.TestPublisherSubscriberAfterShutdown.test_processes_finished_gracefully[UnboundedSequences]

Issue #26 in nightly_win_rep
- Failing 15.38% in 2 out of 13 builds
	rclpy.test_service.xunit.missing_result

Issue #27 in nightly_win_rep
- Failing 15.38% in 2 out of 13 builds
	rclcpp_action.TestServer.publish_status_aborted
	rclcpp_action.TestServer.publish_feedback
	rclcpp_action.TestServer.get_result_deferred
	rclcpp_action.TestServer.get_result
	rclcpp_action.TestServer.deferred_execution
	projectroot.test_server

Issue #28 in gz_sim-ci-ign-gazebo3-focal-amd64
- Failing 25.0% in 1 out of 4 builds
	CameraSensorBackgroundFixture.RedBackground

Issue #29 in gz_sim-ci-ign-gazebo6-homebrew-amd64
- Failing 16.67% in 1 out of 6 builds
	ServerRepeat/OdometryPublisherTest.GaussianNoiseTest/0

Issue #30 in nightly_linux-rhel_repeated
- Failing 6.25% in 1 out of 16 builds
	rclcpp.TestTimerCancelBehavior/MultiThreadedExecutor.testBothTimerCancelThenResetT2Behavior
	rclcpp.TestAddCallbackGroupsToExecutorStable/MultiThreadedExecutor.subscriber_triggered_to_receive_message

Issue #31 in nightly_linux-rhel_repeated
- Failing 6.25% in 1 out of 16 builds
	projectroot.test.rclcpp.test_executors_timer_cancel_behavior
	  (also in: "nightly_win_rep" job)
	  (also in: "nightly_win_rel" job)

Issue #32 in nightly_win_rep
- Failing 7.69% in 1 out of 13 builds
	projectroot.test_publisher (known issue in: https://github.com/ros2/system_tests/issues/496 - CLOSED)
	  (also in: "Rci__nightly-fastrtps-dynamic_ubuntu_noble_amd64" job) (known issue in: https://github.com/ros2/system_tests/issues/496 - CLOSED)
	  (also in: "Jci__nightly-fastrtps-dynamic_ubuntu_noble_amd64" job) (known issue in: https://github.com/ros2/system_tests/issues/496 - CLOSED)
	  (also in: "Ici__nightly-fastrtps-dynamic_ubuntu_jammy_amd64" job) (known issue in: https://github.com/ros2/system_tests/issues/496 - CLOSED)
	  (also in: "Ici__nightly-connext_ubuntu_jammy_amd64" job) (known issue in: https://github.com/ros2/system_tests/issues/496 - CLOSED)

Issue #33 in nightly_win_rep
- Failing 7.69% in 1 out of 13 builds
	rclpy.test_service_introspection.xunit.missing_result
	rclpy.test_rosout_subscription.xunit.missing_result
	rclpy.test_rate.xunit.missing_result
	rclpy.test_qos_overriding_options.xunit.missing_result
	rclpy.test_qos_event.xunit.missing_result
	rclpy.test_publisher.xunit.missing_result
	rclcpp_action.TestServer.publish_status_succeeded
	rclcpp_action.TestServer.publish_status_canceled
	rclcpp.TestClockSleep.sleep_until_basic_steady
	projectroot.test_service_introspection
	projectroot.test_rosout_subscription
	projectroot.test_rate
	projectroot.test_qos_overriding_options
	projectroot.test_qos_event
	projectroot.point_cloud_transport_qos_override
	point_cloud_transport.TestQosOverride.qos_override_subscriber_without_options
	point_cloud_transport.TestQosOverride.qos_override_subscriber_with_options
	point_cloud_transport.TestQosOverride.qos_override_publisher_without_options
	point_cloud_transport.TestQosOverride.qos_override_publisher_with_options

@Crola1702
Copy link
Contributor

Crola1702 commented Jun 13, 2024

Ros2 network issues last night

Gazebo network issues last night

Copy link
Contributor Author

Jun 14

Build regressions
Jci__nightly-fastrtps-dynamic_ubuntu_noble_amd64|22|2024-06-14 09:40:14.991000|Jenkins channel closing down (Agent disconnected)|build workspace in isolation and install
Test regressions
Excluding 'performance, connext, fastrtps-dynamic' jobs
Issue #1 in nightly_linux_coverage
- Failing 100.0% in 15 out of 15 builds
	rosidl_typesupport_introspection_tests.MultiNestedMessageIntrospectionTest/0.CanWriteTypeErasedMessage
	projectroot.test_multi_nested_message_introspection
	  (also in: "nightly_win_rep" job)
	  (also in: "nightly_linux_coverage" job)

Issue #2 in Rci__benchmark_ubuntu_noble_amd64
- Failing 100.0% in 11 out of 11 builds
	projectroot.benchmark_string_conversions
	  (also in: "Jci__benchmark_ubuntu_noble_amd64" job)

Issue #3 in nightly_win_deb
- Failing 57.14% in 8 out of 14 builds
	sros2.test.sros2.commands.security.verbs.test_generate_policy_no_nodes.test_generate_policy_no_nodes
	  (also in: "nightly_win_rel" job)
	  (also in: "nightly_win_deb" job)

Issue #4 in nightly_linux_repeated
- Failing 43.75% in 7 out of 16 builds
	rclcpp.TestClockWakeup.multiple_threads_wait_on_one_clock
	projectroot.test.rclcpp.test_clock
	  (also in: "nightly_win_rep" job)
	  (also in: "nightly_linux_repeated" job)
	  (also in: "nightly_linux-rhel_repeated" job)

Issue #5 in nightly_linux_debug
- Failing 37.5% in 6 out of 16 builds
	demo_nodes_cpp.TestExecutablesTutorial.test_processes_output
	  (also in: "nightly_win_rep" job)
	  (also in: "nightly_win_rel" job)
	  (also in: "nightly_win_deb" job)
	  (also in: "nightly_linux_repeated" job)
	  (also in: "nightly_linux_debug" job)
	  (also in: "nightly_linux-rhel_repeated" job)
	  (also in: "nightly_linux-rhel_debug" job)

Issue #6 in Jci__nightly-cyclonedds_ubuntu_noble_amd64
- Failing 100.0% in 5 out of 5 builds
	projectroot.test_lifecycle_service_client (known issue in: https://github.com/ros2/rclcpp/issues/2472 - OPEN)
	  (also in: "nightly_win_rep" job) (known issue in: https://github.com/ros2/rclcpp/issues/2472 - OPEN)
	  (also in: "nightly_win_rel" job) (known issue in: https://github.com/ros2/rclcpp/issues/2472 - OPEN)
	  (also in: "Rci__nightly-cyclonedds_ubuntu_noble_amd64" job) (known issue in: https://github.com/ros2/rclcpp/issues/2472 - OPEN)

Issue #7 in Jci__nightly-cyclonedds_ubuntu_noble_amd64
- Failing 100.0% in 5 out of 5 builds
	rclcpp.TestExecutors/StaticSingleThreadedExecutor.spin_some (known issue in: https://github.com/ros2/rclcpp/issues/2502 - OPEN)
	rclcpp.TestExecutors/SingleThreadedExecutor.spin_some_max_duration
	rclcpp.TestExecutors/SingleThreadedExecutor.spin_some (known issue in: https://github.com/ros2/rclcpp/issues/2502 - OPEN)
	rclcpp.TestExecutors/MultiThreadedExecutor.spin_some_max_duration
	rclcpp.TestExecutors/MultiThreadedExecutor.spin_some (known issue in: https://github.com/ros2/rclcpp/issues/2502 - OPEN)
	  (also in: "nightly_win_rep" job) (known issue in: https://github.com/ros2/rclcpp/issues/2502 - OPEN)
	  (also in: "Rci__nightly-cyclonedds_ubuntu_noble_amd64" job) (known issue in: https://github.com/ros2/rclcpp/issues/2502 - OPEN)
	  (also in: "Jci__nightly-cyclonedds_ubuntu_noble_amd64" job)

Issue #8 in Jci__nightly-cyclonedds_ubuntu_noble_amd64
- Failing 100.0% in 5 out of 5 builds
	rclpy.rclpy.test.test_guard_condition.TestGuardCondition.test_trigger (known issue in: https://github.com/ros2/rclpy/issues/1266 - OPEN)
	rclpy.rclpy.test.test_guard_condition.TestGuardCondition.test_double_trigger (known issue in: https://github.com/ros2/rclpy/issues/1266 - OPEN)
	rclcpp_lifecycle.TestLifecycleServiceClientRCLErrors.call_services_rcl_errors (known issue in: https://github.com/ros2/rclcpp/issues/2472 - OPEN)
	rclcpp.TestBusyWaiting/StaticSingleThreadedExecutor.test_spin_some
	rclcpp.TestBusyWaiting/SingleThreadedExecutor.test_spin_some
	rclcpp.TestBusyWaiting/SingleThreadedExecutor.test_spin_all
	rclcpp.TestBusyWaiting/MultiThreadedExecutor.test_spin_some
	rclcpp.TestBusyWaiting/MultiThreadedExecutor.test_spin_all
	projectroot.test_guard_condition (known issue in: https://github.com/ros2/rclpy/issues/1266 - OPEN)
	  (also in: "Rci__nightly-cyclonedds_ubuntu_noble_amd64" job) (known issue in: https://github.com/ros2/rclpy/issues/1266 - OPEN)
	  (also in: "Jci__nightly-cyclonedds_ubuntu_noble_amd64" job)

Issue #9 in gz_rendering-ci-main-noble-amd64
- Failing 100.0% in 5 out of 5 builds
	INTEGRATION_load_unload_ogre_gl3plus.test_ran

Issue #10 in nightly_win_deb
- Failing 35.71% in 5 out of 14 builds
	test_rclcpp.test_multithreaded.multi_access_publisher
	  (also in: "nightly_win_rep" job)

Issue #11 in Jci__nightly-cyclonedds_ubuntu_noble_amd64
- Failing 100.0% in 5 out of 5 builds
	projectroot.test.rclcpp.test_executors (known issue in: https://github.com/ros2/rclcpp/issues/2502 - OPEN)
	  (also in: "nightly_win_rep" job) (known issue in: https://github.com/ros2/rclcpp/issues/2502 - OPEN)
	  (also in: "nightly_win_rel" job) (known issue in: https://github.com/ros2/rclcpp/issues/2502 - OPEN)
	  (also in: "Rci__nightly-cyclonedds_ubuntu_noble_amd64" job) (known issue in: https://github.com/ros2/rclcpp/issues/2502 - OPEN)
	  (also in: "Jci__nightly-cyclonedds_ubuntu_noble_amd64" job) (known issue in: https://github.com/ros2/rclcpp/issues/2502 - OPEN)
	  (also in: "Ici__nightly-connext_ubuntu_jammy_amd64" job) (known issue in: https://github.com/ros2/rclcpp/issues/2502 - OPEN)

Issue #12 in gz_rendering-7-win
- Failing 100.0% in 4 out of 4 builds
	MeshTest.NormalMapWithoutTexCoord
	  (also in: "gz_rendering-main-win" job)
	  (also in: "gz_rendering-8-win" job)
	  (also in: "gz_rendering-7-win" job)

Issue #13 in gz_sensors-ci-main-noble-amd64
- Failing 100.0% in 4 out of 4 builds
	BoundingBoxCameraSensor/BoundingBoxCameraSensorTest.BoxesWithBuiltinSDF/ogre2
	<init>.BoundingBoxCameraSensor/BoundingBoxCameraSensorTest

Issue #14 in nightly_win_rep
- Failing 33.33% in 4 out of 12 builds
	rclpy.test_subscription.xunit.missing_result
	  (also in: "nightly_win_rel" job)

Issue #15 in nightly_win_rep
- Failing 33.33% in 4 out of 12 builds
	projectroot.test_subscription
	  (also in: "nightly_win_rel" job)
	  (also in: "Rci__nightly-fastrtps-dynamic_ubuntu_noble_amd64" job)
	  (also in: "Jci__nightly-fastrtps-dynamic_ubuntu_noble_amd64" job)
	  (also in: "Ici__nightly-fastrtps-dynamic_ubuntu_jammy_amd64" job)
	  (also in: "Ici__nightly-connext_ubuntu_jammy_amd64" job)

Issue #16 in nightly_win_rep
- Failing 33.33% in 4 out of 12 builds
	projectroot.test_publisher_subscriber__rclcpp__rmw_fastrtps_cpp

Issue #17 in nightly_linux-rhel_repeated
- Failing 18.75% in 3 out of 16 builds
	ros2cli.ros2cli.test.test_strategy.test_with_daemon_spawn[1-3]
	  (also in: "nightly_linux_repeated" job)

Issue #18 in nightly_linux_iron_coverage
- Failing 20.0% in 3 out of 15 builds
	launch_testing_ros.test.examples.wait_for_topic_launch_test.wait_for_topic_launch_test (known issue in: https://github.com/ros2/launch_ros/issues/304 - CLOSED)
	  (also in: "nightly_linux_humble_coverage" job) (known issue in: https://github.com/ros2/launch_ros/issues/304 - CLOSED)
	  (also in: "Ici__nightly-debug_ubuntu_jammy_amd64" job) (known issue in: https://github.com/ros2/launch_ros/issues/304 - CLOSED)

Issue #19 in nightly_win_rep
- Failing 25.0% in 3 out of 12 builds
	rclpy.test_waitable.xunit.missing_result
	projectroot.test_waitable
	  (also in: "nightly_win_rel" job)
	  (also in: "nightly_linux-aarch64_repeated" job)

Issue #20 in nightly_win_rep
- Failing 25.0% in 3 out of 12 builds
	projectroot.test_timer
	  (also in: "nightly_win_rel" job)
	  (also in: "nightly_linux_iron_coverage" job)

Issue #21 in nightly_win_rep
- Failing 25.0% in 3 out of 12 builds
	rclpy.test_wait_for_message.xunit.missing_result
	rclpy.test_timer.xunit.missing_result
	rclpy.test_time_source.xunit.missing_result
	projectroot.test_wait_for_message
	projectroot.test_time_source
	  (also in: "nightly_win_rel" job)

Issue #22 in nightly_win_rep
- Failing 25.0% in 3 out of 12 builds
	projectroot.test_service

Issue #23 in Rci__nightly-cyclonedds_ubuntu_noble_amd64
- Failing 16.67% in 2 out of 12 builds
	rclcpp.TestBusyWaiting/MultiThreadedExecutor.test_spin
	  (also in: "Jci__nightly-cyclonedds_ubuntu_noble_amd64" job)

Issue #24 in gz_sim-7-win
- Failing 33.33% in 2 out of 6 builds
	INTEGRATION_save_world.test_ran (known issue in: https://github.com/gazebosim/gz-sim/issues/2177 - CLOSED)
	  (also in: "gz_sim-main-win" job) (known issue in: https://github.com/gazebosim/gz-sim/issues/2177 - CLOSED)
	  (also in: "gz_sim-8-win" job) (known issue in: https://github.com/gazebosim/gz-sim/issues/2177 - CLOSED)
	  (also in: "gz_sim-7-win" job) (known issue in: https://github.com/gazebosim/gz-sim/issues/2177 - CLOSED)

Issue #25 in nightly_win_rel
- Failing 12.5% in 2 out of 16 builds
	rclcpp.TestServiceIntrospection.service_introspection_enable_disable_events
	  (also in: "Rci__nightly-connext_ubuntu_noble_amd64" job)
	  (also in: "Jci__nightly-connext_ubuntu_noble_amd64" job)
	  (also in: "Ici__nightly-connext_ubuntu_jammy_amd64" job)

Issue #26 in nightly_win_rel
- Failing 12.5% in 2 out of 16 builds
	rclcpp.TestServiceIntrospection.service_introspection_enable_disable_event_content
	  (also in: "Rci__nightly-connext_ubuntu_noble_amd64" job)
	  (also in: "Jci__nightly-connext_ubuntu_noble_amd64" job)
	  (also in: "Ici__nightly-connext_ubuntu_jammy_amd64" job)

Issue #27 in nightly_win_rel
- Failing 12.5% in 2 out of 16 builds
	rclcpp.TestService.server_qos
	  (also in: "Rci__nightly-connext_ubuntu_noble_amd64" job)
	  (also in: "Jci__nightly-connext_ubuntu_noble_amd64" job)
	  (also in: "Ici__nightly-connext_ubuntu_jammy_amd64" job)

Issue #28 in nightly_win_rel
- Failing 12.5% in 2 out of 16 builds
	rclcpp.TestQosEvent.test_pub_matched_event_by_option_event_callback
	  (also in: "Ici__nightly-connext_ubuntu_jammy_amd64" job)

Issue #29 in nightly_win_rel
- Failing 12.5% in 2 out of 16 builds
	rclcpp.TestQosEvent.test_on_new_event_callback
	rclcpp.TestQosEvent.test_invalid_on_new_event_callback
	  (also in: "Hci__nightly-connext_ubuntu_jammy_amd64" job)

Issue #30 in nightly_win_rel
- Failing 12.5% in 2 out of 16 builds
	projectroot.test.rclcpp.test_subscription_topic_statistics
	  (also in: "nightly_win_rep" job)
	  (also in: "Rci__nightly-connext_ubuntu_noble_amd64" job)
	  (also in: "Jci__nightly-connext_ubuntu_noble_amd64" job)

Issue #31 in nightly_win_rel
- Failing 12.5% in 2 out of 16 builds
	rclcpp.TestSubscription.on_new_message_callback
	rclcpp.TestService.on_new_request_callback
	projectroot.test.rclcpp.test_subscription
	  (also in: "Ici__nightly-connext_ubuntu_jammy_amd64" job)
	  (also in: "Hci__nightly-connext_ubuntu_jammy_amd64" job)

Issue #32 in nightly_win_rel
- Failing 12.5% in 2 out of 16 builds
	projectroot.test.rclcpp.test_service_introspection
	  (also in: "Rci__nightly-connext_ubuntu_noble_amd64" job)
	  (also in: "Jci__nightly-connext_ubuntu_noble_amd64" job)
	  (also in: "Ici__nightly-connext_ubuntu_jammy_amd64" job)

Issue #33 in nightly_win_rel
- Failing 12.5% in 2 out of 16 builds
	projectroot.test.rclcpp.test_service
	  (also in: "Rci__nightly-connext_ubuntu_noble_amd64" job)
	  (also in: "Jci__nightly-connext_ubuntu_noble_amd64" job)
	  (also in: "Ici__nightly-connext_ubuntu_jammy_amd64" job)
	  (also in: "Hci__nightly-connext_ubuntu_jammy_amd64" job)

Issue #34 in nightly_win_rel
- Failing 12.5% in 2 out of 16 builds
	rclcpp.TestInterfaceTraits.has_node_base_interface
	rclcpp.TestExecutors/SingleThreadedExecutor.detachOnDestruction
	projectroot.test.rclcpp.test_logger_service
	projectroot.test.rclcpp.test_interface_traits
	  (also in: "nightly_win_rep" job)

Issue #35 in nightly_win_rel
- Failing 12.5% in 2 out of 16 builds
	rclcpp.test_intra_process_within_one_node.type_adapted_messages_unique_pub_received_by_intra_process_subscription
	rclcpp.test_intra_process_within_one_node.type_adapted_messages_unique_ptr_ros_message_pub_received_by_intra_process_subscription
	rclcpp.test_intra_process_within_one_node.type_adapted_messages_ros_message_ref_pub_received_by_intra_process_subscription
	rclcpp.test_intra_process_within_one_node.type_adapted_messages_const_ref_pub_received_by_intra_process_subscription
	rclcpp.TestWaitForAllAckedWithParm/TestPublisherWaitForAllAcked.check_wait_for_all_acked_with_QosPolicy/2 ((88-byte object <01-00 00-00 00-00 00-00 01-00 00-00 00-00 00-00 01-00 00-00 02-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 
	rclcpp.TestWaitForAllAckedWithParm/TestPublisherWaitForAllAcked.check_wait_for_all_acked_with_QosPolicy/1 ((88-byte object <01-00 00-00 00-00 00-00 01-00 00-00 00-00 00-00 02-00 00-00 02-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 
	rclcpp.TestWaitForAllAckedWithParm/TestPublisherWaitForAllAcked.check_wait_for_all_acked_with_QosPolicy/0 ((88-byte object <01-00 00-00 00-00 00-00 01-00 00-00 00-00 00-00 01-00 00-00 02-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 
	rclcpp.TestTypeSupport.test_service_ts_set_params_srv
	rclcpp.TestTypeSupport.test_service_ts_set_params_atomically_srv
	rclcpp.TestTypeSupport.test_service_ts_list_params_srv
	rclcpp.TestTypeSupport.test_service_ts_get_params_srv_type
	rclcpp.TestTypeSupport.test_service_ts_get_params_srv
	rclcpp.TestTypeSupport.test_service_ts_get_parameters_types_srv
	rclcpp.TestTypeSupport.test_service_ts_describe_params_srv
	rclcpp.TestTypeSupport.basic_getters
	rclcpp.TestSubscriptionPublisherCount/one_context_test.increasing_and_decreasing_counts
	rclcpp.TestSubscription.various_creation_signatures
	rclcpp.TestSubscription.take_serialized
	rclcpp.TestSubscription.take
	rclcpp.TestSubscription.rcl_take_type_erased_error
	rclcpp.TestSubscription.rcl_take_serialized_message_error
	rclcpp.TestSubscription.rcl_subscription_init_error
	rclcpp.TestSubscription.rcl_subscription_get_publisher_count_error
	rclcpp.TestSubscription.rcl_subscription_get_actual_qos_error
	rclcpp.TestSubscription.rcl_subscription_fini_error
	rclcpp.TestSubscription.on_new_intra_process_message_callback
	rclcpp.TestSubscription.handle_loaned_message
	rclcpp.TestSubscription.get_network_flow_endpoints_errors
	rclcpp.TestSubscription.construction_and_destruction
	rclcpp.TestSubscription.check_type_adapted_messages_are_received_by_intra_process_subscription
	rclcpp.TestSubscription.check_type_adapted_messages_are_received_by_inter_process_subscription
	rclcpp.TestSubscription.callback_bind
	rclcpp.TestServiceIntrospection.service_introspection_nominal
	rclcpp.TestService.take_request
	rclcpp.TestService.server_qos_depth
	rclcpp.TestService.send_response
	rclcpp.TestService.rcl_service_response_publisher_get_actual_qos_error
	rclcpp.TestService.rcl_service_request_subscription_get_actual_qos_error
	rclcpp.TestService.construction_and_destruction_rcl_errors
	rclcpp.TestService.construction_and_destruction
	rclcpp.TestService.basic_public_getters
	rclcpp.TestSerializedMessageAllocator.borrow_from_subscription
	rclcpp.TestQosParameters.declare_with_id
	rclcpp.TestQosParameters.declare_with_callback
	rclcpp.TestQosParameters.declare_qos_subscription_parameters
	rclcpp.TestQosParameters.declare_no_parameters_interface
	rclcpp.TestQosParameters.declare
	rclcpp.TestQosEvent.test_subscription_constructor
	rclcpp.TestQosEvent.test_sub_matched_event_by_set_event_callback
	rclcpp.TestQosEvent.test_sub_matched_event_by_option_event_callback
	rclcpp.TestQosEvent.test_publisher_constructor
	rclcpp.TestQosEvent.test_pub_matched_event_by_set_event_callback
	rclcpp.TestQosEvent.test_default_incompatible_qos_callbacks
	rclcpp.TestQosEvent.execute
	rclcpp.TestQosEvent.construct_destruct_rcl_error
	rclcpp.TestQosEvent.add_to_wait_set
	rclcpp.TestPublisherThrows/TestPublisherInvalidIntraprocessQos.test_publisher_throws/keep_all_qos (keep_all_qos)                     
	rclcpp.TestPublisherSubscriptionCount/two_subscriptions_intraprocess_comm.increasing_and_decreasing_counts
	rclcpp.TestPublisherSub.construction_and_destruction
	rclcpp.TestPublisher.various_creation_signatures
	rclcpp.TestPublisher.test_large_message_unique
	rclcpp.TestPublisher.test_large_message_constref
	rclcpp.TestPublisher.serialized_message_publish
	rclcpp.TestPublisher.run_event_handlers
	rclcpp.TestPublisher.rcl_publisher_init_error
	rclcpp.TestPublisher.rcl_publisher_get_subscription_count_publisher_invalid
	rclcpp.TestPublisher.rcl_publisher_get_rmw_handle_error
	rclcpp.TestPublisher.rcl_publisher_get_options_error
	rclcpp.TestPublisher.rcl_publisher_get_gid_for_publisher_error
	rclcpp.TestPublisher.rcl_publisher_get_actual_qos_error
	rclcpp.TestPublisher.rcl_publisher_fini_error
	rclcpp.TestPublisher.publishers_equal_rmw_compare_gids_error
	rclcpp.TestPublisher.lowest_available_ipm_capacity
	rclcpp.TestPublisher.intra_process_transient_local
	rclcpp.TestPublisher.intra_process_publish_failures
	rclcpp.TestPublisher.inter_process_publish_failures
	rclcpp.TestPublisher.get_network_flow_endpoints_errors
	rclcpp.TestPublisher.do_loaned_message_publish_error
	rclcpp.TestPublisher.default_incompatible_qos_callback
	rclcpp.TestPublisher.conversion_exception_is_passed_up
	rclcpp.TestPublisher.construction_and_destruction
	rclcpp.TestPublisher.check_wait_for_all_acked_return
	rclcpp.TestPublisher.check_type_adapted_message_is_sent_and_received
	rclcpp.TestPublisher.basic_getters
	rclcpp.TestParameterService.set_parameters_atomically
	rclcpp.TestParameterService.set_parameters
	rclcpp.TestParameterService.list_parameters
	rclcpp.TestParameterService.get_parameters
	rclcpp.TestParameterService.get_parameter_types
	rclcpp.TestParameterService.describe_parameters
	rclcpp.TestParameterClient.sync_parameter_load_parameters
	rclcpp.TestParameterClient.sync_parameter_is_ready
	rclcpp.TestParameterClient.sync_parameter_get_parameters_allow_undeclared
	rclcpp.TestParameterClient.sync_parameter_get_parameters
	rclcpp.TestParameterClient.sync_parameter_get_parameter_types_allow_undeclared
	rclcpp.TestParameterClient.sync_parameter_get_parameter_types
	rclcpp.TestParameterClient.sync_parameter_get_parameter
	rclcpp.TestParameterClient.sync_parameter_event_subscription
	rclcpp.TestParameterClient.sync_parameter_describe_parameters_allow_undeclared
	rclcpp.TestParameterClient.sync_parameter_describe_parameters
	rclcpp.TestParameterClient.sync_parameter_delete_parameters
	rclcpp.TestParameterClient.sync_construction_and_destruction
	rclcpp.TestParameterClient.async_parameter_set_parameters_atomically
	rclcpp.TestParameterClient.async_parameter_load_parameters_from_map
	rclcpp.TestParameterClient.async_parameter_load_parameters_complicated_regex
	rclcpp.TestParameterClient.async_parameter_load_parameters
	rclcpp.TestParameterClient.async_parameter_load_no_valid_parameter
	rclcpp.TestParameterClient.async_parameter_load_from_map_no_valid_parameter
	rclcpp.TestParameterClient.async_parameter_list_parameters
	rclcpp.TestParameterClient.async_parameter_get_parameters_allow_undeclared
	rclcpp.TestParameterClient.async_parameter_get_parameters
	rclcpp.TestParameterClient.async_parameter_get_parameter_types_allow_undeclared
	rclcpp.TestParameterClient.async_parameter_get_parameter_types
	rclcpp.TestParameterClient.async_parameter_event_subscription
	rclcpp.TestParameterClient.async_parameter_describe_parameters_allow_undeclared
	rclcpp.TestParameterClient.async_parameter_describe_parameters
	rclcpp.TestParameterClient.async_parameter_delete_parameters
	rclcpp.TestParameterClient.async_construction_and_destruction
	rclcpp.TestNodeWithGlobalArgs.use_or_ignore_global_arguments
	rclcpp.TestNodeWithGlobalArgs.local_arguments_before_global
	rclcpp.TestNodeWaitables.add_remove_waitable
	rclcpp.TestNodeTypeDescriptions.interface_created
	rclcpp.TestNodeTopics.add_publisher
	rclcpp.TestNodeTimers.add_timer
	rclcpp.TestNodeService.add_service
	rclcpp.TestNodeParameters.construct_destruct_rcl_errors
	rclcpp.TestNodeInterfaces.node_interfaces_standard_interfaces
	rclcpp.TestNodeInterfaces.node_interfaces_nominal
	rclcpp.TestNodeInterfaces.ni_init
	rclcpp.TestNodeInterfaces.ni_all_init
	rclcpp.TestNodeInterfaces.default_constructor
	rclcpp.TestNode.SameParameterDifferentNode
	rclcpp.TestNode.RegisterParameterCallback
	rclcpp.TestNode.MultipleParameterCallbacks
	rclcpp.TestNode.LastInFirstCallForParameterEventCallbacks
	rclcpp.TestNode.LastInFirstCallForParameterCallbacks
	rclcpp.TestNode.GetParametersFromEvent
	rclcpp.TestNode.GetParameterFromEvent
	rclcpp.TestNode.EventCallback
	rclcpp.TestLogger.get_node_logger
	rclcpp.TestFindWeakNodes.allocator_strategy_with_weak_nodes
	rclcpp.TestFindWeakNodes.allocator_strategy_no_weak_nodes
	rclcpp.TestExternallyDefinedServices.extern_defined_uninitialized
	rclcpp.TestExternallyDefinedServices.extern_defined_initialized
	rclcpp.TestExternallyDefinedServices.extern_defined_destructor
	rclcpp.TestExternallyDefinedServices.default_behavior
	rclcpp.TestClockSleep.bad_clock_type
	rclcpp.RclcppGenericNodeFixture.publisher_and_subscriber_work
	rclcpp.RclcppGenericNodeFixture.publish_loaned_msg_work
	rclcpp.RclcppGenericNodeFixture.generic_subscription_uses_qos
	rclcpp.RclcppGenericNodeFixture.generic_subscription_different_callbacks
	rclcpp.RclcppGenericNodeFixture.generic_publisher_uses_qos
	projectroot.test.rclcpp.test_type_support
	projectroot.test.rclcpp.test_subscription_with_type_adapter
	projectroot.test.rclcpp.test_subscription_publisher_with_same_type_adapter
	projectroot.test.rclcpp.test_subscription_publisher_count_api
	projectroot.test.rclcpp.test_serialized_message_allocator
	projectroot.test.rclcpp.test_qos_parameters
	projectroot.test.rclcpp.test_qos_event__rmw_fastrtps_cpp
	projectroot.test.rclcpp.test_publisher_with_type_adapter
	projectroot.test.rclcpp.test_publisher_subscription_count_api (known issue in: https://github.com/ros2/rclcpp/issues/2230 - CLOSED)
	projectroot.test.rclcpp.test_publisher (known issue in: https://github.com/ros2/rclcpp/issues/2230 - CLOSED)
	projectroot.test.rclcpp.test_parameter_service
	projectroot.test.rclcpp.test_parameter_event_handler
	projectroot.test.rclcpp.test_parameter_client
	projectroot.test.rclcpp.test_node_interfaces__node_waitables
	projectroot.test.rclcpp.test_node_interfaces__node_type_descriptions
	projectroot.test.rclcpp.test_node_interfaces__node_topics
	projectroot.test.rclcpp.test_node_interfaces__node_timers
	projectroot.test.rclcpp.test_node_interfaces__node_services
	projectroot.test.rclcpp.test_node_interfaces__node_parameters
	projectroot.test.rclcpp.test_node_interfaces__node_interfaces
	projectroot.test.rclcpp.test_node_global_args
	projectroot.test.rclcpp.test_logger
	projectroot.test.rclcpp.test_generic_pubsub__rmw_fastrtps_cpp
	projectroot.test.rclcpp.test_find_weak_nodes
	projectroot.test.rclcpp.test_externally_defined_services

Issue #36 in nightly_win_rel
- Failing 12.5% in 2 out of 16 builds
	projectroot.test.rclcpp.test_executors_timer_cancel_behavior
	  (also in: "nightly_win_rep" job)
	  (also in: "nightly_linux-rhel_repeated" job)

Issue #37 in nightly_win_rel
- Failing 12.5% in 2 out of 16 builds
	rclcpp.test_executor.gtest.missing_result
	projectroot.test.rclcpp.test_executor (known issue in: https://github.com/ros2/rclcpp/issues/2502 - OPEN)
	  (also in: "nightly_win_rep" job)

Issue #38 in nightly_win_rel
- Failing 12.5% in 2 out of 16 builds
	rclcpp.TestWaitSet.add_remove_wait
	rclcpp.TestWaitSet.add_guard_condition_to_two_different_wait_set
	rclcpp.TestUtilities.wait_for_message_twice_one_sub
	rclcpp.TestUtilities.wait_for_message_indefinitely
	rclcpp.TestUtilities.wait_for_message
	rclcpp.TestTimerCancelBehavior/SingleThreadedExecutor.testTimer2CancelledWithExecutorSpin
	rclcpp.TestTimerCancelBehavior/SingleThreadedExecutor.testTimer1CancelledWithExecutorSpin
	rclcpp.TestTimerCancelBehavior/SingleThreadedExecutor.testHeadTimerCancelThenResetBehavior
	rclcpp.TestTimerCancelBehavior/SingleThreadedExecutor.testBothTimerCancelThenResetT2Behavior
	rclcpp.TestTimerCancelBehavior/SingleThreadedExecutor.testBothTimerCancelThenResetT1Behavior
	rclcpp.TestTimerCancelBehavior/SingleThreadedExecutor.testBackTimerCancelThenResetBehavior
	rclcpp.TestTimeSource.reattach
	rclcpp.TestTimeSource.parameter_activation
	rclcpp.TestTimeSource.no_pre_jump_callback
	rclcpp.TestTimeSource.detachUnattached
	rclcpp.TestTimeSource.clock_sleep_until_with_ros_time_basic
	rclcpp.TestTimeSource.clock
	rclcpp.TestTimeSource.check_use_clock_thread_value
	rclcpp.TestTimeSource.check_clock_thread_status
	rclcpp.TestTimeSource.callbacks
	rclcpp.TestTimeSource.callback_handler_erasure
	rclcpp.TestTimeSource.ROS_time_valid_wall_time
	rclcpp.TestTimeSource.ROS_time_valid_sim_time
	rclcpp.TestTimeSource.ROS_time_valid_attach_detach
	rclcpp.TestTimeSource.ROS_invalid_sim_time
	rclcpp.TestThreadSafeStorage.wait_waitable
	rclcpp.TestThreadSafeStorage.wait_timer
	rclcpp.TestThreadSafeStorage.wait_subscription
	rclcpp.TestThreadSafeStorage.wait_client_service
	rclcpp.TestThreadSafeStorage.iterables_construct_destruct
	rclcpp.TestThreadSafeStorage.default_construct_destruct
	rclcpp.TestThreadSafeStorage.add_remove_out_of_scope
	rclcpp.TestThreadSafeStorage.add_remove_nullptr
	rclcpp.TestThreadSafeStorage.add_remove_dynamically
	rclcpp.TestSubscriptionTopicStatisticsFixture.test_invalid_publish_period
	rclcpp.TestSubscriptionOptions.topic_statistics_options_node_default_mode
	rclcpp.TestStoragePolicyCommon.rcl_wait_set_resize_error
	rclcpp.TestStoragePolicyCommon.rcl_wait_set_fini_error
	rclcpp.TestStoragePolicyCommon.rcl_wait_set_clear_error
	rclcpp.TestStoragePolicyCommon.rcl_wait_set_add_timer_error
	rclcpp.TestStoragePolicyCommon.rcl_wait_set_add_subscription_error
	rclcpp.TestStoragePolicyCommon.rcl_wait_set_add_service_error
	rclcpp.TestStoragePolicyCommon.rcl_wait_set_add_guard_condition_error
	rclcpp.TestStoragePolicyCommon.rcl_wait_set_add_client_error
	rclcpp.TestStoragePolicyCommon.add_waitable_error
	rclcpp.TestStaticStorage.wait_waitable
	rclcpp.TestStaticStorage.wait_timer
	rclcpp.TestStaticStorage.wait_subscription
	rclcpp.TestStaticStorage.wait_guard_condition
	rclcpp.TestStaticStorage.wait_client_service
	rclcpp.TestStaticStorage.iterables_construct_destruct
	rclcpp.TestStaticStorage.fixed_storage_needs_pruning
	rclcpp.TestStaticSingleThreadedExecutor.add_callback_group_trigger_guard_failed
	rclcpp.TestRosoutSubscription.test_rosoutsubscription_parent_log
	rclcpp.TestRosoutSubscription.test_rosoutsubscription_node_rosout_disabled
	rclcpp.TestRosoutSubscription.test_rosoutsubscription_getchild_hierarchy
	rclcpp.TestRosoutSubscription.test_rosoutsubscription_getchild
	rclcpp.TestRosoutSubscription.test_rosoutsubscription_child_log
	rclcpp.TestMultiThreadedExecutor.timer_over_take
	rclcpp.TestLoggerService.check_connect_get_logger_service
	rclcpp.TestIntraprocessExecutors/SingleThreadedExecutor.testIntraprocessRetrigger
	rclcpp.TestIntraProcessWaitable.test_that_waitable_stays_ready_after_second_wait
	rclcpp.TestGraphListener.construction_and_destruction
	rclcpp.TestExecutorNotifyWaitable.add_remove_guard_conditions
	rclcpp.TestExecutorEntitiesCollector.add_remove_node
	rclcpp.TestEventsExecutor.run_pub_sub
	rclcpp.TestDynamicStorage.wait_waitable
	rclcpp.TestDynamicStorage.wait_timer
	rclcpp.TestDynamicStorage.wait_subscription
	rclcpp.TestDynamicStorage.wait_client_service
	rclcpp.TestDynamicStorage.iterables_construct_destruct
	rclcpp.TestDynamicStorage.default_construct_destruct
	rclcpp.TestDynamicStorage.add_remove_out_of_scope
	rclcpp.TestDynamicStorage.add_remove_nullptr
	rclcpp.TestDynamicStorage.add_remove_dynamically
	rclcpp.TestContentFilterSubscription.set_content_filter_error
	rclcpp.TestContentFilterSubscription.set_content_filter
	rclcpp.TestContentFilterSubscription.is_cft_enabled
	rclcpp.TestContentFilterSubscription.get_content_filter_error
	rclcpp.TestContentFilterSubscription.get_content_filter
	rclcpp.TestContentFilterSubscription.content_filter_reset
	rclcpp.TestContentFilterSubscription.content_filter_get_later
	rclcpp.TestContentFilterSubscription.content_filter_get_begin
	rclcpp.TestCallbackGroup.valid_callback_group
	rclcpp.TestCallbackGroup.invalid_callback_group
	rclcpp.PerTimerType/TestTimer.test_simple_cancel/wall_timer (4-byte object <00-00 00-00>)                     
	projectroot.test.rclcpp.test_wait_set
	projectroot.test.rclcpp.test_wait_for_message
	projectroot.test.rclcpp.test_timer (known issue in: https://github.com/ros2/rclcpp/issues/2436 - CLOSED)
	projectroot.test.rclcpp.test_time_source
	projectroot.test.rclcpp.test_thread_safe_synchronization
	projectroot.test.rclcpp.test_subscription_options
	projectroot.test.rclcpp.test_subscription_content_filter__rmw_fastrtps_cpp
	projectroot.test.rclcpp.test_storage_policy_common
	projectroot.test.rclcpp.test_static_storage
	projectroot.test.rclcpp.test_static_single_threaded_executor
	projectroot.test.rclcpp.test_rosout_subscription
	projectroot.test.rclcpp.test_multi_threaded_executor
	projectroot.test.rclcpp.test_intra_process_waitable
	projectroot.test.rclcpp.test_graph_listener
	projectroot.test.rclcpp.test_executors_intraprocess
	projectroot.test.rclcpp.test_executors_callback_group_behavior
	projectroot.test.rclcpp.test_executor_notify_waitable
	projectroot.test.rclcpp.test_events_executor
	projectroot.test.rclcpp.test_entities_collector
	projectroot.test.rclcpp.test_dynamic_storage
	  (also in: "nightly_win_rep" job)

Issue #39 in nightly_win_rep
- Failing 16.67% in 2 out of 12 builds
	test_communication.TestPublisherSubscriberAfterShutdown.test_processes_finished_gracefully[UnboundedSequences]

Issue #40 in nightly_win_rep
- Failing 16.67% in 2 out of 12 builds
	rclpy.test_service.xunit.missing_result

Issue #41 in nightly_win_rep
- Failing 16.67% in 2 out of 12 builds
	rclcpp_action.TestServer.publish_status_aborted
	rclcpp_action.TestServer.publish_feedback
	rclcpp_action.TestServer.get_result_deferred
	rclcpp_action.TestServer.get_result
	rclcpp_action.TestServer.deferred_execution
	projectroot.test_server

Issue #42 in nightly_win_rel
- Failing 12.5% in 2 out of 16 builds
	projectroot.test.rclcpp.test_time (known issue in: https://github.com/ros2/rclcpp/issues/2436 - CLOSED)
	  (also in: "nightly_win_rep" job) (known issue in: https://github.com/ros2/rclcpp/issues/2436 - CLOSED)
	  (also in: "nightly_win_rel" job) (known issue in: https://github.com/ros2/rclcpp/issues/2436 - CLOSED)

Issue #43 in gz_sim-ci-ign-gazebo3-focal-amd64
- Failing 33.33% in 1 out of 3 builds
	CameraSensorBackgroundFixture.RedBackground

Issue #44 in gz_sim-ign-gazebo6-win
- Failing 16.67% in 1 out of 6 builds
	INTEGRATION_multiple_servers.test_ran
	  (also in: "gz_sim-main-win" job)
	  (also in: "gz_sim-8-win" job)

Issue #45 in gz_sim-main-win
- Failing 14.29% in 1 out of 7 builds
	ServerRepeat/MultipleServers.TwoServersMixedBlocking/0

Issue #46 in nightly_linux-aarch64_repeated
- Failing 6.25% in 1 out of 16 builds
	test_rclcpp.TestTwoExecutablesAfterShutdown.test_client_scope_cpp
	  (also in: "nightly_linux_repeated" job)
	  (also in: "nightly_linux-rhel_repeated" job)

Issue #47 in nightly_linux-aarch64_repeated
- Failing 6.25% in 1 out of 16 builds
	projectroot.test_client_scope_cpp__rmw_cyclonedds_cpp
	  (also in: "nightly_linux_repeated" job)
	  (also in: "nightly_linux-rhel_repeated" job)

Issue #48 in nightly_linux-aarch64_repeated
- Failing 6.25% in 1 out of 16 builds
	projectroot.src.core.ddsc.tests.CUnit_ddsc_discstress_create_writer

Issue #49 in nightly_linux-rhel_repeated
- Failing 6.25% in 1 out of 16 builds
	rosbag2_py.test_transport_py.xunit.missing_result
	rclcpp.TestAddCallbackGroupsToExecutorStable/EventsExecutor.subscriber_triggered_to_receive_message
	projectroot.test_transport_py

Issue #50 in nightly_linux_debug
- Failing 6.25% in 1 out of 16 builds
	ros2cli.ros2cli.test.test_strategy.test_with_daemon_running
	  (also in: "nightly_win_rel" job)

Issue #51 in nightly_linux_debug
- Failing 6.25% in 1 out of 16 builds
	ros2cli.ros2cli.test.test_strategy.test_with_daemon_spawn
	ros2cli.ros2cli.test.test_strategy.test_enforce_no_daemon

Issue #52 in nightly_win_rel
- Failing 6.25% in 1 out of 16 builds
	rclcpp.TestContentFilterSubscription.create_two_content_filters_with_same_topic_name_and_destroy

Issue #53 in nightly_win_rep
- Failing 8.33% in 1 out of 12 builds
	projectroot.test_publisher (known issue in: https://github.com/ros2/system_tests/issues/496 - CLOSED)
	  (also in: "Rci__nightly-fastrtps-dynamic_ubuntu_noble_amd64" job) (known issue in: https://github.com/ros2/system_tests/issues/496 - CLOSED)
	  (also in: "Jci__nightly-fastrtps-dynamic_ubuntu_noble_amd64" job) (known issue in: https://github.com/ros2/system_tests/issues/496 - CLOSED)
	  (also in: "Ici__nightly-fastrtps-dynamic_ubuntu_jammy_amd64" job) (known issue in: https://github.com/ros2/system_tests/issues/496 - CLOSED)
	  (also in: "Ici__nightly-connext_ubuntu_jammy_amd64" job) (known issue in: https://github.com/ros2/system_tests/issues/496 - CLOSED)

Issue #54 in nightly_win_rep
- Failing 8.33% in 1 out of 12 builds
	rclpy.test_service_introspection.xunit.missing_result
	rclpy.test_rosout_subscription.xunit.missing_result
	rclpy.test_rate.xunit.missing_result
	rclpy.test_qos_overriding_options.xunit.missing_result
	rclpy.test_qos_event.xunit.missing_result
	rclpy.test_publisher.xunit.missing_result
	rclcpp_action.TestServer.publish_status_succeeded
	rclcpp_action.TestServer.publish_status_canceled
	rclcpp.TestClockSleep.sleep_until_basic_steady
	projectroot.test_service_introspection
	projectroot.test_rosout_subscription
	projectroot.test_rate
	projectroot.test_qos_overriding_options
	projectroot.test_qos_event
	projectroot.point_cloud_transport_qos_override
	point_cloud_transport.TestQosOverride.qos_override_subscriber_without_options
	point_cloud_transport.TestQosOverride.qos_override_subscriber_with_options
	point_cloud_transport.TestQosOverride.qos_override_publisher_without_options
	point_cloud_transport.TestQosOverride.qos_override_publisher_with_options

@Crola1702
Copy link
Contributor

Crola1702 commented Jun 14, 2024

Nightly Linux Coverage | Test regression | Investigating 🕵️

Reference build: https://ci.ros2.org/job/nightly_linux_coverage/2426/

Test regressions:

Log output:

[ RUN      ] MultiNestedMessageIntrospectionTest/0.CanWriteTypeErasedMessage
/home/jenkins-agent/workspace/nightly_linux_coverage/ws/src/ros2/rosidl/rosidl_typesupport_introspection_tests/test/test_multi_nested_message_introspection.cpp:212: Failure
Expected equality of these values:
  item_reference
    Which is: 736-byte object <90-B1 3D-F2 2A-7F 00-00 90-00 00-00 2A-7F 00-00 F0-A7 01-BB 09-56 00-00 F0-A7 01-BB 09-56 00-00 00-00 00-01 0C-00 00-00 35-3C 2E-75 B6-AC 9F-D8 91-85 71-8B 00-00 A1-01 F0-FF FF-FF 00-00 00-01 ... 01-00 00-00 00-00 00-00 60-30 02-BB 09-56 00-00 09-00 00-00 00-00 00-00 0A-00 00-00 00-00 00-00 80-30 02-BB 09-56 00-00 09-00 00-00 00-00 00-00 0A-00 00-00 00-00 00-00 00-00 A1-01 00-00 00-00>
  message.array_of_arrays[i]
    Which is: 736-byte object <90-B1 3D-F2 2A-7F 00-00 90-B1 3D-F2 2A-7F 00-00 F0-A7 01-BB 09-56 00-00 F0-A7 01-BB 09-56 00-00 00-00 00-01 0C-00 00-00 35-3C 2E-75 B6-AC 9F-D8 91-85 71-8B 00-00 A1-01 F0-FF FF-FF 00-00 00-01 ... 01-00 00-00 00-00 00-00 00-C3 01-BB 09-56 00-00 09-00 00-00 00-00 00-00 0A-00 00-00 00-00 00-00 E0-C2 01-BB 09-56 00-00 09-00 00-00 00-00 00-00 0A-00 00-00 00-00 00-00 00-00 A1-01 10-00 00-00>

/home/jenkins-agent/workspace/nightly_linux_coverage/ws/src/ros2/rosidl/rosidl_typesupport_introspection_tests/test/test_multi_nested_message_introspection.cpp:256: Failure
Expected equality of these values:
  message
    Which is: 6864-byte object <90-B1 3D-F2 2A-7F 00-00 90-B1 3D-F2 2A-7F 00-00 F0-A7 01-BB 09-56 00-00 F0-A7 01-BB 09-56 00-00 00-00 00-01 0C-00 00-00 35-3C 2E-75 B6-AC 9F-D8 91-85 71-8B 00-00 A1-01 F0-FF FF-FF 00-00 00-01 ... 01-00 00-00 00-00 00-00 01-00 00-00 00-00 00-00 20-FA 01-BB 09-56 00-00 01-00 00-00 00-00 00-00 01-00 00-00 00-00 00-00 70-FF 01-BB 09-56 00-00 01-00 00-00 00-00 00-00 01-00 00-00 00-00 00-00>
  message_copy
    Which is: 6864-byte object <90-B1 3D-F2 2A-7F 00-00 90-00 00-00 2A-7F 00-00 F0-A7 01-BB 09-56 00-00 F0-A7 01-BB 09-56 00-00 00-00 00-01 0C-00 00-00 35-3C 2E-75 B6-AC 9F-D8 91-85 71-8B 00-00 A1-01 F0-FF FF-FF 00-00 00-01 ... 01-00 00-00 00-00 00-00 01-00 00-00 00-00 00-00 70-85 02-BB 09-56 00-00 01-00 00-00 00-00 00-00 01-00 00-00 00-00 00-00 10-8D 02-BB 09-56 00-00 01-00 00-00 00-00 00-00 01-00 00-00 00-00 00-00>

[  FAILED  ] MultiNestedMessageIntrospectionTest/0.CanWriteTypeErasedMessage, where TypeParam = rosidl_typesupport_introspection_tests__msg__MultiNested (1 ms)

Reported in: ros2/rosidl#812

Copy link
Contributor Author

Jun 15

Build regressions
Rci__nightly-connext_ubuntu_noble_amd64|59|2024-06-15 06:53:10.100000|Jenkins channel closing down (Agent disconnected)|run tests
Rci__nightly-connext_ubuntu_noble_amd64|58|2024-06-15 02:20:58.297000|Number of failed test exceeds the threshold|Clean up to save disk space on agents
Rci__nightly-connext_ubuntu_noble_amd64|57|2024-06-14 13:03:05.440000|Unknown Reason|Clean up to save disk space on agents
Rci__nightly-fastrtps-dynamic_ubuntu_noble_amd64|56|2024-06-14 17:42:07.917000|Number of failed test exceeds the threshold|run tests
sdformat-sdf12-win|52|2024-06-14 16:06:34.005000|Jenkins channel closing down (Agent disconnected)|configure the MSVC compiler
Test regressions
Excluding 'performance, connext, fastrtps-dynamic' jobs
Issue #1 in nightly_linux_coverage
- Failing 100.0% in 15 out of 15 builds
	rosidl_typesupport_introspection_tests.MultiNestedMessageIntrospectionTest/0.CanWriteTypeErasedMessage
	projectroot.test_multi_nested_message_introspection
	  (also in: "nightly_win_rep" job)

Issue #2 in nightly_win_deb
- Failing 40.0% in 6 out of 15 builds
	test_rclcpp.test_multithreaded.multi_access_publisher
	  (also in: "nightly_win_rep" job)

Issue #3 in Jci__nightly-cyclonedds_ubuntu_noble_amd64
- Failing 100.0% in 5 out of 5 builds
	projectroot.test_lifecycle_service_client (known issue in: https://github.com/ros2/rclcpp/issues/2472 - OPEN)
	  (also in: "nightly_win_rep" job) (known issue in: https://github.com/ros2/rclcpp/issues/2472 - OPEN)
	  (also in: "nightly_win_rel" job) (known issue in: https://github.com/ros2/rclcpp/issues/2472 - OPEN)
	  (also in: "Rci__nightly-cyclonedds_ubuntu_noble_amd64" job) (known issue in: https://github.com/ros2/rclcpp/issues/2472 - OPEN)

Issue #4 in Jci__nightly-cyclonedds_ubuntu_noble_amd64
- Failing 100.0% in 5 out of 5 builds
	rclcpp.TestExecutors/StaticSingleThreadedExecutor.spin_some (known issue in: https://github.com/ros2/rclcpp/issues/2502 - OPEN)
	rclcpp.TestExecutors/SingleThreadedExecutor.spin_some_max_duration
	rclcpp.TestExecutors/SingleThreadedExecutor.spin_some (known issue in: https://github.com/ros2/rclcpp/issues/2502 - OPEN)
	rclcpp.TestExecutors/MultiThreadedExecutor.spin_some_max_duration
	rclcpp.TestExecutors/MultiThreadedExecutor.spin_some (known issue in: https://github.com/ros2/rclcpp/issues/2502 - OPEN)
	  (also in: "nightly_win_rep" job) (known issue in: https://github.com/ros2/rclcpp/issues/2502 - OPEN)
	  (also in: "Rci__nightly-cyclonedds_ubuntu_noble_amd64" job) (known issue in: https://github.com/ros2/rclcpp/issues/2502 - OPEN)
	  (also in: "Jci__nightly-cyclonedds_ubuntu_noble_amd64" job)

Issue #5 in Jci__nightly-cyclonedds_ubuntu_noble_amd64
- Failing 100.0% in 5 out of 5 builds
	rclpy.rclpy.test.test_guard_condition.TestGuardCondition.test_trigger (known issue in: https://github.com/ros2/rclpy/issues/1266 - OPEN)
	rclpy.rclpy.test.test_guard_condition.TestGuardCondition.test_double_trigger (known issue in: https://github.com/ros2/rclpy/issues/1266 - OPEN)
	rclcpp_lifecycle.TestLifecycleServiceClientRCLErrors.call_services_rcl_errors (known issue in: https://github.com/ros2/rclcpp/issues/2472 - OPEN)
	rclcpp.TestBusyWaiting/StaticSingleThreadedExecutor.test_spin_some
	rclcpp.TestBusyWaiting/SingleThreadedExecutor.test_spin_some
	rclcpp.TestBusyWaiting/MultiThreadedExecutor.test_spin_some
	rclcpp.TestBusyWaiting/MultiThreadedExecutor.test_spin_all
	projectroot.test_guard_condition (known issue in: https://github.com/ros2/rclpy/issues/1266 - OPEN)
	  (also in: "Rci__nightly-cyclonedds_ubuntu_noble_amd64" job) (known issue in: https://github.com/ros2/rclpy/issues/1266 - OPEN)
	  (also in: "Jci__nightly-cyclonedds_ubuntu_noble_amd64" job)

Issue #6 in gz_rendering-ci-main-noble-amd64
- Failing 100.0% in 5 out of 5 builds
	INTEGRATION_load_unload_ogre_gl3plus.test_ran

Issue #7 in Jci__nightly-cyclonedds_ubuntu_noble_amd64
- Failing 100.0% in 5 out of 5 builds
	rclcpp.TestBusyWaiting/SingleThreadedExecutor.test_spin_all
	  (also in: "nightly_win_rep" job)
	  (also in: "Rci__nightly-cyclonedds_ubuntu_noble_amd64" job)
	  (also in: "Jci__nightly-cyclonedds_ubuntu_noble_amd64" job)

Issue #8 in Jci__nightly-cyclonedds_ubuntu_noble_amd64
- Failing 100.0% in 5 out of 5 builds
	projectroot.test.rclcpp.test_executors (known issue in: https://github.com/ros2/rclcpp/issues/2502 - OPEN)
	  (also in: "nightly_win_rep" job) (known issue in: https://github.com/ros2/rclcpp/issues/2502 - OPEN)
	  (also in: "nightly_win_rel" job) (known issue in: https://github.com/ros2/rclcpp/issues/2502 - OPEN)
	  (also in: "Rci__nightly-cyclonedds_ubuntu_noble_amd64" job) (known issue in: https://github.com/ros2/rclcpp/issues/2502 - OPEN)
	  (also in: "Jci__nightly-cyclonedds_ubuntu_noble_amd64" job) (known issue in: https://github.com/ros2/rclcpp/issues/2502 - OPEN)
	  (also in: "Ici__nightly-connext_ubuntu_jammy_amd64" job) (known issue in: https://github.com/ros2/rclcpp/issues/2502 - OPEN)

Issue #9 in gz_rendering-7-win
- Failing 100.0% in 4 out of 4 builds
	MeshTest.NormalMapWithoutTexCoord
	  (also in: "gz_rendering-main-win" job)
	  (also in: "gz_rendering-8-win" job)
	  (also in: "gz_rendering-7-win" job)

Issue #10 in gz_sensors-ci-main-noble-amd64
- Failing 100.0% in 4 out of 4 builds
	BoundingBoxCameraSensor/BoundingBoxCameraSensorTest.BoxesWithBuiltinSDF/ogre2
	<init>.BoundingBoxCameraSensor/BoundingBoxCameraSensorTest

Issue #11 in nightly_linux_coverage
- Failing 26.67% in 4 out of 15 builds
	ros2topic.ros2topic.test.test_use_sim_time.test_use_sim_time (known issue in: https://github.com/ros2/ros2cli/issues/772 - CLOSED)

Issue #12 in nightly_linux_iron_coverage
- Failing 26.67% in 4 out of 15 builds
	launch_testing_ros.test.examples.wait_for_topic_launch_test.wait_for_topic_launch_test (known issue in: https://github.com/ros2/launch_ros/issues/304 - CLOSED)
	  (also in: "nightly_linux_humble_coverage" job) (known issue in: https://github.com/ros2/launch_ros/issues/304 - CLOSED)
	  (also in: "Ici__nightly-debug_ubuntu_jammy_amd64" job) (known issue in: https://github.com/ros2/launch_ros/issues/304 - CLOSED)

Issue #13 in Jci__benchmark_ubuntu_noble_amd64
- Failing 100.0% in 3 out of 3 builds
	projectroot.benchmark_string_conversions
	  (also in: "Rci__benchmark_ubuntu_noble_amd64" job)
	  (also in: "Jci__benchmark_ubuntu_noble_amd64" job)

Issue #14 in nightly_linux_repeated
- Failing 18.75% in 3 out of 16 builds
	ros2param.ros2param.test.test_verb_load.test_verb_load (known issue in: https://github.com/ros2/rmw_fastrtps/issues/531 - CLOSED | https://github.com/ros2/ros2cli/issues/861 - CLOSED)

Issue #15 in nightly_linux_repeated
- Failing 18.75% in 3 out of 16 builds
	test_rclcpp.test_node_name__rmw_fastrtps_cpp.xunit.missing_result
	projectroot.test_node_name__rmw_fastrtps_cpp

Issue #16 in nightly_win_rep
- Failing 25.0% in 3 out of 12 builds
	test_cli_remapping.TestCLIRemappingAfterShutdown.test_processes_finished_gracefully[C:/ci/ws/build/test_cli_remapping/Release/name_maker_rclcpp.exe]

Issue #17 in nightly_win_rep
- Failing 25.0% in 3 out of 12 builds
	rclcpp_action.TestServer.publish_status_aborted
	rclcpp_action.TestServer.publish_feedback
	rclcpp_action.TestServer.get_result_deferred
	rclcpp_action.TestServer.get_result
	rclcpp_action.TestServer.deferred_execution
	projectroot.test_server

Issue #18 in nightly_win_rep
- Failing 25.0% in 3 out of 12 builds
	projectroot.test_client
	  (also in: "Rci__nightly-connext_ubuntu_noble_amd64" job)
	  (also in: "Jci__nightly-connext_ubuntu_noble_amd64" job)
	  (also in: "Hci__nightly-connext_ubuntu_jammy_amd64" job)

Issue #19 in nightly_win_rep
- Failing 25.0% in 3 out of 12 builds
	projectroot.test_cli_remapping
	  (also in: "nightly_linux_repeated" job)
	  (also in: "nightly_linux-rhel_repeated" job)
	  (also in: "nightly_linux-aarch64_repeated" job)

Issue #20 in nightly_linux_repeated
- Failing 18.75% in 3 out of 16 builds
	demo_nodes_cpp.TestExecutablesTutorial.test_processes_output
	  (also in: "nightly_win_rep" job)
	  (also in: "nightly_win_rel" job)
	  (also in: "nightly_win_deb" job)
	  (also in: "nightly_linux_repeated" job)
	  (also in: "nightly_linux_debug" job)
	  (also in: "nightly_linux-rhel_repeated" job)
	  (also in: "nightly_linux-rhel_debug" job)

Issue #21 in gz_sim-7-win
- Failing 33.33% in 2 out of 6 builds
	INTEGRATION_save_world.test_ran (known issue in: https://github.com/gazebosim/gz-sim/issues/2177 - CLOSED)
	  (also in: "gz_sim-main-win" job) (known issue in: https://github.com/gazebosim/gz-sim/issues/2177 - CLOSED)
	  (also in: "gz_sim-8-win" job) (known issue in: https://github.com/gazebosim/gz-sim/issues/2177 - CLOSED)
	  (also in: "gz_sim-7-win" job) (known issue in: https://github.com/gazebosim/gz-sim/issues/2177 - CLOSED)

Issue #22 in nightly_linux-rhel_repeated
- Failing 12.5% in 2 out of 16 builds
	test_rclcpp.TestTwoExecutablesAfterShutdown.test_client_scope_cpp
	  (also in: "nightly_linux_repeated" job)
	  (also in: "nightly_linux-rhel_repeated" job)
	  (also in: "nightly_linux-aarch64_repeated" job)

Issue #23 in nightly_linux_repeated
- Failing 12.5% in 2 out of 16 builds
	ros2topic.ros2topic.test.test_cli.test_cli (known issue in: https://github.com/ros2/ros2cli/issues/861 - CLOSED)
	  (also in: "nightly_linux-aarch64_repeated" job) (known issue in: https://github.com/ros2/ros2cli/issues/861 - CLOSED)

Issue #24 in nightly_win_rep
- Failing 16.67% in 2 out of 12 builds
	rclcpp_action.TestServer.publish_status_succeeded
	rclcpp_action.TestServer.publish_status_canceled

Issue #25 in gz_sensors-ci-ign-sensors6-homebrew-amd64
- Failing 25.0% in 1 out of 4 builds
	CameraSensor/TriggeredCameraTest.ImagesWithBuiltinSDF/ogre

Issue #26 in gz_sim-ign-gazebo6-win
- Failing 20.0% in 1 out of 5 builds
	INTEGRATION_multiple_servers.test_ran
	  (also in: "gz_sim-main-win" job)
	  (also in: "gz_sim-8-win" job)

Issue #27 in gz_sim-main-win
- Failing 16.67% in 1 out of 6 builds
	ServerRepeat/MultipleServers.TwoServersMixedBlocking/0

Issue #28 in gz_transport-ci-gz-transport13-homebrew-amd64
- Failing 16.67% in 1 out of 6 builds
	gzTest.TopicEchoNum

Issue #29 in nightly_linux_repeated
- Failing 6.25% in 1 out of 16 builds
	test_security.TestSecurePublisherSubscriberAfterShutdown.test_processes_finished_gracefully
	  (also in: "nightly_win_rep" job)
	  (also in: "nightly_linux_release" job)

Issue #30 in nightly_linux_repeated
- Failing 6.25% in 1 out of 16 builds
	test_security.test_secure_publisher_subscriber__UnboundedSequences__rmw_cyclonedds_cpp__secure_comm_2.xunit.missing_result
	test_security.TestSecurePublisherSubscriber.test_subscriber_terminates_in_a_finite_amount_of_time
	projectroot.test_secure_publisher_subscriber__UnboundedSequences__rmw_cyclonedds_cpp__secure_not_connecting_1
	projectroot.test_secure_publisher_subscriber__UnboundedSequences__rmw_cyclonedds_cpp__secure_not_connecting_0
	projectroot.test_secure_publisher_subscriber__UnboundedSequences__rmw_cyclonedds_cpp__secure_comm_3
	projectroot.test_secure_publisher_subscriber__UnboundedSequences__rmw_cyclonedds_cpp__secure_comm_2
	projectroot.test_secure_publisher_subscriber__UnboundedSequences__rmw_cyclonedds_cpp__secure_comm_1
	projectroot.test_secure_publisher_subscriber__UnboundedSequences__rmw_cyclonedds_cpp__secure_comm_0
	projectroot.test_secure_publisher_subscriber__Empty__rmw_cyclonedds_cpp__secure_not_connecting_1
	projectroot.test_secure_publisher_subscriber__Empty__rmw_cyclonedds_cpp__secure_not_connecting_0
	projectroot.test_secure_publisher_subscriber__Empty__rmw_cyclonedds_cpp__secure_comm_3
	projectroot.test_secure_publisher_subscriber__Empty__rmw_cyclonedds_cpp__secure_comm_2
	projectroot.test_secure_publisher_subscriber__Empty__rmw_cyclonedds_cpp__secure_comm_1
	projectroot.test_secure_publisher_subscriber__Empty__rmw_cyclonedds_cpp__secure_comm_0
	  (also in: "nightly_linux_release" job)

Issue #31 in nightly_win_rel
- Failing 6.25% in 1 out of 16 builds
	test_communication.TestPublisherSubscriberAfterShutdown.test_processes_finished_gracefully[UnboundedSequences]
	  (also in: "nightly_win_rep" job)

Issue #32 in nightly_win_rel
- Failing 6.25% in 1 out of 16 builds
	test_communication.TestPublisherSubscriberAfterShutdown.test_processes_finished_gracefully[Empty] (known issue in: https://github.com/ros2/system_tests/issues/496 - CLOSED)
	  (also in: "nightly_win_rep" job) (known issue in: https://github.com/ros2/system_tests/issues/496 - CLOSED)
	  (also in: "nightly_linux_repeated" job) (known issue in: https://github.com/ros2/system_tests/issues/496 - CLOSED)

Issue #33 in nightly_win_rel
- Failing 6.25% in 1 out of 16 builds
	test_communication.TestPublisherSubscriberAfterShutdown.test_processes_finished_gracefully[Constants] (known issue in: https://github.com/ros2/system_tests/issues/496 - CLOSED)
	  (also in: "nightly_win_rep" job) (known issue in: https://github.com/ros2/system_tests/issues/496 - CLOSED)
	  (also in: "nightly_linux_repeated" job) (known issue in: https://github.com/ros2/system_tests/issues/496 - CLOSED)

Issue #34 in nightly_win_rel
- Failing 6.25% in 1 out of 16 builds
	test_communication.TestPublisherSubscriberAfterShutdown.test_processes_finished_gracefully[BoundedPlainSequences] (known issue in: https://github.com/ros2/system_tests/issues/496 - CLOSED)
	  (also in: "nightly_win_rep" job) (known issue in: https://github.com/ros2/system_tests/issues/496 - CLOSED)

Issue #35 in nightly_win_rel
- Failing 6.25% in 1 out of 16 builds
	test_communication.test_publisher_subscriber_serialized__rmw_fastrtps_cpp.gtest.missing_result
	projectroot.test_publisher_subscriber_serialized__rmw_fastrtps_cpp

Issue #36 in nightly_win_rel
- Failing 6.25% in 1 out of 16 builds
	projectroot.test_publisher_subscriber__rclcpp__rmw_fastrtps_cpp
	  (also in: "nightly_win_rep" job)

Issue #37 in nightly_win_rel
- Failing 6.25% in 1 out of 16 builds
	test_communication.TestRequesterReplierAfterShutdown.test_processes_finished_gracefully[Empty]
	test_communication.TestRequesterReplierAfterShutdown.test_processes_finished_gracefully[BasicTypes]
	test_communication.TestRequesterReplierAfterShutdown.test_processes_finished_gracefully[Arrays]
	test_communication.TestRequesterReplier.test_requester_finishes_in_a_finite_amount_of_time[Empty]
	test_communication.TestRequesterReplier.test_requester_finishes_in_a_finite_amount_of_time[BasicTypes]
	test_communication.TestRequesterReplier.test_requester_finishes_in_a_finite_amount_of_time[Arrays]
	test_communication.TestPublisherSubscriberAfterShutdown.test_processes_finished_gracefully[WStrings] (known issue in: https://github.com/ros2/system_tests/issues/496 - CLOSED)
	test_communication.TestPublisherSubscriberAfterShutdown.test_processes_finished_gracefully[Strings] (known issue in: https://github.com/ros2/system_tests/issues/496 - CLOSED)
	test_communication.TestPublisherSubscriberAfterShutdown.test_processes_finished_gracefully[Nested]
	test_communication.TestPublisherSubscriberAfterShutdown.test_processes_finished_gracefully[MultiNested] (known issue in: https://github.com/ros2/system_tests/issues/496 - CLOSED)
	test_communication.TestPublisherSubscriberAfterShutdown.test_processes_finished_gracefully[Defaults]
	test_communication.TestPublisherSubscriberAfterShutdown.test_processes_finished_gracefully[Builtins]
	test_communication.TestPublisherSubscriberAfterShutdown.test_processes_finished_gracefully[BoundedSequences]
	test_communication.TestPublisherSubscriberAfterShutdown.test_processes_finished_gracefully[BasicTypes] (known issue in: https://github.com/ros2/system_tests/issues/496 - CLOSED)
	test_communication.TestPublisherSubscriberAfterShutdown.test_processes_finished_gracefully[Arrays]
	test_communication.TestPublisherSubscriber.test_subscriber_terminates_in_a_finite_amount_of_time[WStrings]
	test_communication.TestPublisherSubscriber.test_subscriber_terminates_in_a_finite_amount_of_time[UnboundedSequences]
	test_communication.TestPublisherSubscriber.test_subscriber_terminates_in_a_finite_amount_of_time[Strings]
	test_communication.TestPublisherSubscriber.test_subscriber_terminates_in_a_finite_amount_of_time[Nested]
	test_communication.TestPublisherSubscriber.test_subscriber_terminates_in_a_finite_amount_of_time[MultiNested]
	test_communication.TestPublisherSubscriber.test_subscriber_terminates_in_a_finite_amount_of_time[Empty] (known issue in: https://github.com/ros2/system_tests/issues/496 - CLOSED)
	test_communication.TestPublisherSubscriber.test_subscriber_terminates_in_a_finite_amount_of_time[Defaults]
	test_communication.TestPublisherSubscriber.test_subscriber_terminates_in_a_finite_amount_of_time[Constants]
	test_communication.TestPublisherSubscriber.test_subscriber_terminates_in_a_finite_amount_of_time[Builtins]
	test_communication.TestPublisherSubscriber.test_subscriber_terminates_in_a_finite_amount_of_time[BoundedSequences]
	test_communication.TestPublisherSubscriber.test_subscriber_terminates_in_a_finite_amount_of_time[BoundedPlainSequences] (known issue in: https://github.com/ros2/system_tests/issues/496 - CLOSED)
	test_communication.TestPublisherSubscriber.test_subscriber_terminates_in_a_finite_amount_of_time[BasicTypes]
	test_communication.TestPublisherSubscriber.test_subscriber_terminates_in_a_finite_amount_of_time[Arrays]
	test_communication.TestActionClientServerAfterShutdown.test_processes_finished_gracefully[NestedMessage]
	test_communication.TestActionClientServerAfterShutdown.test_processes_finished_gracefully[Fibonacci]
	test_communication.TestActionClientServer.test_client_finishes_in_a_finite_amount_of_time[NestedMessage]
	test_communication.TestActionClientServer.test_client_finishes_in_a_finite_amount_of_time[Fibonacci]
	projectroot.test_requester_replier__rclpy__rmw_fastrtps_cpp
	projectroot.test_requester_replier__rclpy__rclcpp__rmw_fastrtps_cpp
	projectroot.test_requester_replier__rclcpp__rmw_fastrtps_cpp
	projectroot.test_requester_replier__rclcpp__rclpy__rmw_fastrtps_cpp
	projectroot.test_publisher_subscriber__rclpy__rmw_fastrtps_cpp (known issue in: https://github.com/ros2/system_tests/issues/496 - CLOSED)
	projectroot.test_publisher_subscriber__rclpy__rclcpp__rmw_fastrtps_cpp
	projectroot.test_publisher_subscriber__rclcpp__rclpy__rmw_fastrtps_cpp (known issue in: https://github.com/ros2/system_tests/issues/496 - CLOSED)
	projectroot.test_action_client_server__rclpy__rmw_fastrtps_cpp
	projectroot.test_action_client_server__rclpy__rclcpp__rmw_fastrtps_cpp
	projectroot.test_action_client_server__rclcpp__rmw_fastrtps_cpp
	projectroot.test_action_client_server__rclcpp__rclpy__rmw_fastrtps_cpp
	  (also in: "nightly_win_rep" job)

Issue #38 in nightly_win_rep
- Failing 8.33% in 1 out of 12 builds
	ros2cli.ros2cli.test.test_strategy.test_with_daemon_spawn[2-3]
	  (also in: "nightly_linux_repeated" job)
	  (also in: "nightly_linux-rhel_repeated" job)
	  (also in: "nightly_linux-aarch64_repeated" job)

Issue #39 in nightly_win_rep
- Failing 8.33% in 1 out of 12 builds
	test_cli_remapping.TestCLIRemapping.test_topic_and_service_replacement[C:/ci/ws/build/test_cli_remapping/Release/name_maker_rclcpp.exe]
	rclcpp_action.TestServer.publish_status_canceling
	rclcpp_action.TestServer.publish_status_accepted
	rclcpp_action.TestServer.handle_goal_called
	rclcpp_action.TestServer.handle_cancel_unknown_goal
	rclcpp_action.TestServer.handle_cancel_terminated_goal
	rclcpp_action.TestServer.handle_cancel_reject
	rclcpp_action.TestServer.handle_cancel_called
	rclcpp_action.TestServer.handle_accepted_called
	rclcpp_action.TestServer.construction_and_destruction_wait_set_error
	rclcpp_action.TestServer.construction_and_destruction_sub_node
	rclcpp_action.TestServer.construction_and_destruction_server_init_error
	rclcpp_action.TestServer.construction_and_destruction_callback_group
	rclcpp_action.TestServer.construction_and_destruction_after_node
	rclcpp_action.TestServer.construction_and_destruction
	rclcpp_action.TestClientAgainstServer.send_rcl_errors
	rclcpp_action.TestClientAgainstServer.execute_rcl_errors
	rclcpp_action.TestClientAgainstServer.deadlock_in_callbacks
	rclcpp_action.TestClientAgainstServer.async_send_goal_with_result_callback_wait_for_result
	rclcpp_action.TestClientAgainstServer.async_send_goal_with_goal_response_callback_wait_for_result
	rclcpp_action.TestClientAgainstServer.async_send_goal_with_feedback_callback_wait_for_result
	rclcpp_action.TestClientAgainstServer.async_get_result_with_callback
	rclcpp_action.TestClientAgainstServer.async_cancel_some_goals_with_callback
	rclcpp_action.TestClientAgainstServer.async_cancel_some_goals
	rclcpp_action.TestClientAgainstServer.async_cancel_one_goal_with_callback
	rclcpp_action.TestClientAgainstServer.async_cancel_one_goal
	rclcpp_action.TestClientAgainstServer.async_cancel_all_goals_with_callback
	rclcpp_action.TestClientAgainstServer.async_cancel_all_goals
	rclcpp.TestExecutors.testSpinUntilFutureCompleteNodePtr
	rclcpp.TestExecutors.testSpinUntilFutureCompleteNodeBasePtr

Copy link
Contributor Author

Jun 16

Build regressions
nightly_linux-aarch64_repeated|2790|2024-06-16 04:00:01.095000|Number of failed test exceeds the threshold|ccache stats (after)
Test regressions
Excluding 'performance, connext, fastrtps-dynamic' jobs
Issue #1 in nightly_linux_coverage
- Failing 100.0% in 15 out of 15 builds
	rosidl_typesupport_introspection_tests.MultiNestedMessageIntrospectionTest/0.CanWriteTypeErasedMessage
	projectroot.test_multi_nested_message_introspection
	  (also in: "nightly_win_rep" job)

Issue #2 in nightly_win_deb
- Failing 53.33% in 8 out of 15 builds
	sros2.test.sros2.commands.security.verbs.test_generate_policy_no_nodes.test_generate_policy_no_nodes
	  (also in: "nightly_win_rel" job)

Issue #3 in nightly_win_deb
- Failing 46.67% in 7 out of 15 builds
	test_rclcpp.test_multithreaded.multi_access_publisher
	  (also in: "nightly_win_rep" job)
	  (also in: "nightly_win_deb" job)

Issue #4 in nightly_linux-rhel_debug
- Failing 37.5% in 6 out of 16 builds
	demo_nodes_cpp.TestExecutablesTutorial.test_processes_output
	  (also in: "nightly_win_rep" job)
	  (also in: "nightly_win_deb" job)
	  (also in: "nightly_linux_repeated" job)
	  (also in: "nightly_linux_debug" job)
	  (also in: "nightly_linux-rhel_repeated" job)
	  (also in: "nightly_linux-rhel_debug" job)

Issue #5 in Jci__nightly-cyclonedds_ubuntu_noble_amd64
- Failing 100.0% in 5 out of 5 builds
	projectroot.test_lifecycle_service_client (known issue in: https://github.com/ros2/rclcpp/issues/2472 - OPEN)
	  (also in: "nightly_win_rep" job) (known issue in: https://github.com/ros2/rclcpp/issues/2472 - OPEN)
	  (also in: "nightly_win_rel" job) (known issue in: https://github.com/ros2/rclcpp/issues/2472 - OPEN)
	  (also in: "Rci__nightly-cyclonedds_ubuntu_noble_amd64" job) (known issue in: https://github.com/ros2/rclcpp/issues/2472 - OPEN)

Issue #6 in Jci__nightly-cyclonedds_ubuntu_noble_amd64
- Failing 100.0% in 5 out of 5 builds
	rclcpp.TestExecutors/StaticSingleThreadedExecutor.spin_some (known issue in: https://github.com/ros2/rclcpp/issues/2502 - OPEN)
	rclcpp.TestExecutors/SingleThreadedExecutor.spin_some_max_duration
	rclcpp.TestExecutors/SingleThreadedExecutor.spin_some (known issue in: https://github.com/ros2/rclcpp/issues/2502 - OPEN)
	rclcpp.TestExecutors/MultiThreadedExecutor.spin_some_max_duration
	rclcpp.TestExecutors/MultiThreadedExecutor.spin_some (known issue in: https://github.com/ros2/rclcpp/issues/2502 - OPEN)
	  (also in: "nightly_win_rep" job) (known issue in: https://github.com/ros2/rclcpp/issues/2502 - OPEN)
	  (also in: "Rci__nightly-cyclonedds_ubuntu_noble_amd64" job) (known issue in: https://github.com/ros2/rclcpp/issues/2502 - OPEN)
	  (also in: "Jci__nightly-cyclonedds_ubuntu_noble_amd64" job)

Issue #7 in Jci__nightly-cyclonedds_ubuntu_noble_amd64
- Failing 100.0% in 5 out of 5 builds
	rclcpp.TestBusyWaiting/SingleThreadedExecutor.test_spin_all
	  (also in: "nightly_win_rep" job)
	  (also in: "Rci__nightly-cyclonedds_ubuntu_noble_amd64" job)
	  (also in: "Jci__nightly-cyclonedds_ubuntu_noble_amd64" job)

Issue #8 in Jci__nightly-cyclonedds_ubuntu_noble_amd64
- Failing 100.0% in 5 out of 5 builds
	rclpy.rclpy.test.test_guard_condition.TestGuardCondition.test_trigger (known issue in: https://github.com/ros2/rclpy/issues/1266 - OPEN)
	rclpy.rclpy.test.test_guard_condition.TestGuardCondition.test_double_trigger (known issue in: https://github.com/ros2/rclpy/issues/1266 - OPEN)
	rclcpp_lifecycle.TestLifecycleServiceClientRCLErrors.call_services_rcl_errors (known issue in: https://github.com/ros2/rclcpp/issues/2472 - OPEN)
	rclcpp.TestBusyWaiting/StaticSingleThreadedExecutor.test_spin_some
	rclcpp.TestBusyWaiting/SingleThreadedExecutor.test_spin_some
	rclcpp.TestBusyWaiting/MultiThreadedExecutor.test_spin_some
	rclcpp.TestBusyWaiting/MultiThreadedExecutor.test_spin_all
	projectroot.test_guard_condition (known issue in: https://github.com/ros2/rclpy/issues/1266 - OPEN)
	  (also in: "Rci__nightly-cyclonedds_ubuntu_noble_amd64" job) (known issue in: https://github.com/ros2/rclpy/issues/1266 - OPEN)
	  (also in: "Jci__nightly-cyclonedds_ubuntu_noble_amd64" job)

Issue #9 in gz_sensors-ci-main-noble-amd64
- Failing 100.0% in 5 out of 5 builds
	BoundingBoxCameraSensor/BoundingBoxCameraSensorTest.BoxesWithBuiltinSDF/ogre2
	<init>.BoundingBoxCameraSensor/BoundingBoxCameraSensorTest

Issue #10 in nightly_linux_coverage
- Failing 33.33% in 5 out of 15 builds
	ros2topic.ros2topic.test.test_use_sim_time.test_use_sim_time (known issue in: https://github.com/ros2/ros2cli/issues/772 - CLOSED)

Issue #11 in Jci__nightly-cyclonedds_ubuntu_noble_amd64
- Failing 100.0% in 5 out of 5 builds
	projectroot.test.rclcpp.test_executors (known issue in: https://github.com/ros2/rclcpp/issues/2502 - OPEN)
	  (also in: "nightly_win_rep" job) (known issue in: https://github.com/ros2/rclcpp/issues/2502 - OPEN)
	  (also in: "nightly_win_rel" job) (known issue in: https://github.com/ros2/rclcpp/issues/2502 - OPEN)
	  (also in: "Rci__nightly-cyclonedds_ubuntu_noble_amd64" job) (known issue in: https://github.com/ros2/rclcpp/issues/2502 - OPEN)
	  (also in: "Jci__nightly-cyclonedds_ubuntu_noble_amd64" job) (known issue in: https://github.com/ros2/rclcpp/issues/2502 - OPEN)
	  (also in: "Ici__nightly-connext_ubuntu_jammy_amd64" job) (known issue in: https://github.com/ros2/rclcpp/issues/2502 - OPEN)

Issue #12 in nightly_win_rep
- Failing 38.46% in 5 out of 13 builds
	rclcpp.TestClockWakeup.multiple_threads_wait_on_one_clock
	projectroot.test.rclcpp.test_clock
	  (also in: "nightly_linux_repeated" job)
	  (also in: "nightly_linux-rhel_repeated" job)

Issue #13 in gz_rendering-ci-main-noble-amd64
- Failing 100.0% in 4 out of 4 builds
	INTEGRATION_load_unload_ogre_gl3plus.test_ran

Issue #14 in gz_rendering-7-win
- Failing 100.0% in 4 out of 4 builds
	MeshTest.NormalMapWithoutTexCoord
	  (also in: "gz_rendering-main-win" job)
	  (also in: "gz_rendering-8-win" job)
	  (also in: "gz_rendering-7-win" job)

Issue #15 in nightly_linux_repeated
- Failing 25.0% in 4 out of 16 builds
	ros2param.ros2param.test.test_verb_load.test_verb_load (known issue in: https://github.com/ros2/rmw_fastrtps/issues/531 - CLOSED | https://github.com/ros2/ros2cli/issues/861 - CLOSED)

Issue #16 in nightly_win_rep
- Failing 30.77% in 4 out of 13 builds
	test_rclcpp.test_timeout_subscriber.timeout_subscriber

Issue #17 in nightly_win_rep
- Failing 30.77% in 4 out of 13 builds
	projectroot.test_client
	  (also in: "Rci__nightly-connext_ubuntu_noble_amd64" job)
	  (also in: "Jci__nightly-connext_ubuntu_noble_amd64" job)
	  (also in: "Hci__nightly-connext_ubuntu_jammy_amd64" job)

Issue #18 in nightly_win_rep
- Failing 30.77% in 4 out of 13 builds
	rclcpp.test_executor.gtest.missing_result
	projectroot.test.rclcpp.test_time (known issue in: https://github.com/ros2/rclcpp/issues/2436 - CLOSED)
	projectroot.test.rclcpp.test_executor (known issue in: https://github.com/ros2/rclcpp/issues/2502 - OPEN)
	  (also in: "nightly_win_rel" job)

Issue #19 in Jci__benchmark_ubuntu_noble_amd64
- Failing 100.0% in 3 out of 3 builds
	projectroot.benchmark_string_conversions
	  (also in: "Rci__benchmark_ubuntu_noble_amd64" job)
	  (also in: "Jci__benchmark_ubuntu_noble_amd64" job)

Issue #20 in nightly_win_rep
- Failing 23.08% in 3 out of 13 builds
	rclcpp.TestInterfaceTraits.has_node_base_interface
	rclcpp.TestExecutors/SingleThreadedExecutor.detachOnDestruction
	projectroot.test.rclcpp.test_logger_service
	projectroot.test.rclcpp.test_interface_traits
	  (also in: "nightly_win_rel" job)

Issue #21 in nightly_win_rep
- Failing 23.08% in 3 out of 13 builds
	demo_nodes_cpp.TestExecutablesTutorialAfterShutdown.test_last_process_exit_code

Issue #22 in gz_sim-7-win
- Failing 40.0% in 2 out of 5 builds
	INTEGRATION_save_world.test_ran (known issue in: https://github.com/gazebosim/gz-sim/issues/2177 - CLOSED)
	  (also in: "gz_sim-main-win" job) (known issue in: https://github.com/gazebosim/gz-sim/issues/2177 - CLOSED)
	  (also in: "gz_sim-8-win" job) (known issue in: https://github.com/gazebosim/gz-sim/issues/2177 - CLOSED)
	  (also in: "gz_sim-7-win" job) (known issue in: https://github.com/gazebosim/gz-sim/issues/2177 - CLOSED)

Issue #23 in nightly_win_rep
- Failing 15.38% in 2 out of 13 builds
	projectroot.test_tutorial_add_two_ints_server_add_two_ints_client_async__rmw_fastrtps_cpp

Issue #24 in nightly_win_rep
- Failing 15.38% in 2 out of 13 builds
	test_communication.TestRequesterReplierAfterShutdown.test_processes_finished_gracefully[Arrays]
	test_communication.TestRequesterReplier.test_requester_finishes_in_a_finite_amount_of_time[Arrays]
	test_communication.TestPublisherSubscriberAfterShutdown.test_processes_finished_gracefully[Arrays]
	test_communication.TestPublisherSubscriber.test_subscriber_terminates_in_a_finite_amount_of_time[Arrays]
	projectroot.test_requester_replier__rclpy__rmw_fastrtps_cpp
	projectroot.test_publisher_subscriber__rclpy__rmw_fastrtps_cpp (known issue in: https://github.com/ros2/system_tests/issues/496 - CLOSED)
	  (also in: "nightly_win_rel" job)

Issue #25 in nightly_win_rep
- Failing 15.38% in 2 out of 13 builds
	projectroot.test.rclcpp.test_subscription_topic_statistics
	  (also in: "nightly_win_rel" job)
	  (also in: "Rci__nightly-connext_ubuntu_noble_amd64" job)
	  (also in: "Jci__nightly-connext_ubuntu_noble_amd64" job)

Issue #26 in nightly_win_rep
- Failing 15.38% in 2 out of 13 builds
	projectroot.test.rclcpp.test_memory_strategy

Issue #27 in nightly_win_rep
- Failing 15.38% in 2 out of 13 builds
	projectroot.test.rclcpp.test_executors_timer_cancel_behavior
	  (also in: "nightly_win_rel" job)
	  (also in: "nightly_linux-rhel_repeated" job)

Issue #28 in nightly_win_rep
- Failing 15.38% in 2 out of 13 builds
	rclcpp.TestWaitSet.add_remove_wait
	rclcpp.TestWaitSet.add_guard_condition_to_two_different_wait_set
	rclcpp.TestUtilities.wait_for_message_twice_one_sub
	rclcpp.TestUtilities.wait_for_message_indefinitely
	rclcpp.TestUtilities.wait_for_message
	rclcpp.TestTimerCancelBehavior/SingleThreadedExecutor.testTimer2CancelledWithExecutorSpin
	rclcpp.TestTimerCancelBehavior/SingleThreadedExecutor.testTimer1CancelledWithExecutorSpin
	rclcpp.TestTimerCancelBehavior/SingleThreadedExecutor.testHeadTimerCancelThenResetBehavior
	rclcpp.TestTimerCancelBehavior/SingleThreadedExecutor.testBothTimerCancelThenResetT2Behavior
	rclcpp.TestTimerCancelBehavior/SingleThreadedExecutor.testBothTimerCancelThenResetT1Behavior
	rclcpp.TestTimerCancelBehavior/SingleThreadedExecutor.testBackTimerCancelThenResetBehavior
	rclcpp.TestTimeSource.reattach
	rclcpp.TestTimeSource.parameter_activation
	rclcpp.TestTimeSource.no_pre_jump_callback
	rclcpp.TestTimeSource.detachUnattached
	rclcpp.TestTimeSource.clock_sleep_until_with_ros_time_basic
	rclcpp.TestTimeSource.clock
	rclcpp.TestTimeSource.check_use_clock_thread_value
	rclcpp.TestTimeSource.check_clock_thread_status
	rclcpp.TestTimeSource.callbacks
	rclcpp.TestTimeSource.callback_handler_erasure
	rclcpp.TestTimeSource.ROS_time_valid_wall_time
	rclcpp.TestTimeSource.ROS_time_valid_sim_time
	rclcpp.TestTimeSource.ROS_time_valid_attach_detach
	rclcpp.TestTimeSource.ROS_invalid_sim_time
	rclcpp.TestThreadSafeStorage.wait_waitable
	rclcpp.TestThreadSafeStorage.wait_timer
	rclcpp.TestThreadSafeStorage.wait_subscription
	rclcpp.TestThreadSafeStorage.wait_client_service
	rclcpp.TestThreadSafeStorage.iterables_construct_destruct
	rclcpp.TestThreadSafeStorage.default_construct_destruct
	rclcpp.TestThreadSafeStorage.add_remove_out_of_scope
	rclcpp.TestThreadSafeStorage.add_remove_nullptr
	rclcpp.TestThreadSafeStorage.add_remove_dynamically
	rclcpp.TestSubscriptionTopicStatisticsFixture.test_invalid_publish_period
	rclcpp.TestSubscriptionOptions.topic_statistics_options_node_default_mode
	rclcpp.TestStoragePolicyCommon.rcl_wait_set_resize_error
	rclcpp.TestStoragePolicyCommon.rcl_wait_set_fini_error
	rclcpp.TestStoragePolicyCommon.rcl_wait_set_clear_error
	rclcpp.TestStoragePolicyCommon.rcl_wait_set_add_timer_error
	rclcpp.TestStoragePolicyCommon.rcl_wait_set_add_subscription_error
	rclcpp.TestStoragePolicyCommon.rcl_wait_set_add_service_error
	rclcpp.TestStoragePolicyCommon.rcl_wait_set_add_guard_condition_error
	rclcpp.TestStoragePolicyCommon.rcl_wait_set_add_client_error
	rclcpp.TestStoragePolicyCommon.add_waitable_error
	rclcpp.TestStaticStorage.wait_waitable
	rclcpp.TestStaticStorage.wait_timer
	rclcpp.TestStaticStorage.wait_subscription
	rclcpp.TestStaticStorage.wait_guard_condition
	rclcpp.TestStaticStorage.wait_client_service
	rclcpp.TestStaticStorage.iterables_construct_destruct
	rclcpp.TestStaticStorage.fixed_storage_needs_pruning
	rclcpp.TestStaticSingleThreadedExecutor.add_callback_group_trigger_guard_failed
	rclcpp.TestRosoutSubscription.test_rosoutsubscription_parent_log
	rclcpp.TestRosoutSubscription.test_rosoutsubscription_node_rosout_disabled
	rclcpp.TestRosoutSubscription.test_rosoutsubscription_getchild_hierarchy
	rclcpp.TestRosoutSubscription.test_rosoutsubscription_getchild
	rclcpp.TestRosoutSubscription.test_rosoutsubscription_child_log
	rclcpp.TestMultiThreadedExecutor.timer_over_take
	rclcpp.TestLoggerService.check_connect_get_logger_service
	rclcpp.TestIntraprocessExecutors/SingleThreadedExecutor.testIntraprocessRetrigger
	rclcpp.TestIntraProcessWaitable.test_that_waitable_stays_ready_after_second_wait
	rclcpp.TestGraphListener.construction_and_destruction
	rclcpp.TestExecutorNotifyWaitable.add_remove_guard_conditions
	rclcpp.TestExecutorEntitiesCollector.add_remove_node
	rclcpp.TestEventsExecutor.run_pub_sub
	rclcpp.TestDynamicStorage.wait_waitable
	rclcpp.TestDynamicStorage.wait_timer
	rclcpp.TestDynamicStorage.wait_subscription
	rclcpp.TestDynamicStorage.wait_client_service
	rclcpp.TestDynamicStorage.iterables_construct_destruct
	rclcpp.TestDynamicStorage.default_construct_destruct
	rclcpp.TestDynamicStorage.add_remove_out_of_scope
	rclcpp.TestDynamicStorage.add_remove_nullptr
	rclcpp.TestDynamicStorage.add_remove_dynamically
	rclcpp.TestContentFilterSubscription.set_content_filter_error
	rclcpp.TestContentFilterSubscription.set_content_filter
	rclcpp.TestContentFilterSubscription.is_cft_enabled
	rclcpp.TestContentFilterSubscription.get_content_filter_error
	rclcpp.TestContentFilterSubscription.get_content_filter
	rclcpp.TestContentFilterSubscription.content_filter_reset
	rclcpp.TestContentFilterSubscription.content_filter_get_later
	rclcpp.TestContentFilterSubscription.content_filter_get_begin
	rclcpp.TestCallbackGroup.valid_callback_group
	rclcpp.TestCallbackGroup.invalid_callback_group
	rclcpp.PerTimerType/TestTimer.test_simple_cancel/wall_timer (4-byte object <00-00 00-00>)                     
	projectroot.test.rclcpp.test_wait_set
	projectroot.test.rclcpp.test_wait_for_message
	projectroot.test.rclcpp.test_timer (known issue in: https://github.com/ros2/rclcpp/issues/2436 - CLOSED)
	projectroot.test.rclcpp.test_time_source
	projectroot.test.rclcpp.test_thread_safe_synchronization
	projectroot.test.rclcpp.test_subscription_options
	projectroot.test.rclcpp.test_subscription_content_filter__rmw_fastrtps_cpp
	projectroot.test.rclcpp.test_storage_policy_common
	projectroot.test.rclcpp.test_static_storage
	projectroot.test.rclcpp.test_static_single_threaded_executor
	projectroot.test.rclcpp.test_rosout_subscription
	projectroot.test.rclcpp.test_multi_threaded_executor
	projectroot.test.rclcpp.test_intra_process_waitable
	projectroot.test.rclcpp.test_graph_listener
	projectroot.test.rclcpp.test_executors_intraprocess
	projectroot.test.rclcpp.test_executors_callback_group_behavior
	projectroot.test.rclcpp.test_executor_notify_waitable
	projectroot.test.rclcpp.test_events_executor
	projectroot.test.rclcpp.test_entities_collector
	projectroot.test.rclcpp.test_dynamic_storage
	  (also in: "nightly_win_rel" job)

Issue #29 in gz_sensors-ci-ign-sensors6-homebrew-amd64
- Failing 25.0% in 1 out of 4 builds
	CameraSensor/TriggeredCameraTest.ImagesWithBuiltinSDF/ogre

Issue #30 in gz_sim-ign-gazebo6-win
- Failing 25.0% in 1 out of 4 builds
	INTEGRATION_multiple_servers.test_ran
	  (also in: "gz_sim-main-win" job)
	  (also in: "gz_sim-8-win" job)

Issue #31 in gz_sim-main-win
- Failing 16.67% in 1 out of 6 builds
	ServerRepeat/MultipleServers.TwoServersMixedBlocking/0

Issue #32 in gz_transport-ci-gz-transport13-homebrew-amd64
- Failing 16.67% in 1 out of 6 builds
	gzTest.TopicEchoNum

Issue #33 in nightly_linux-aarch64_release
- Failing 6.25% in 1 out of 16 builds
	launch_testing_examples.launch_testing_examples.hello_world_launch_test.launch_testing_examples.hello_world_launch_test
	  (also in: "nightly_linux-rhel_repeated" job)
	  (also in: "nightly_linux-aarch64_repeated" job)

Issue #34 in nightly_linux-rhel_repeated
- Failing 6.25% in 1 out of 16 builds
	test_rclcpp.TestTwoExecutablesAfterShutdown.test_client_scope_consistency_cpp

Issue #35 in nightly_linux_repeated
- Failing 6.25% in 1 out of 16 builds
	test_communication.TestPublisherSubscriberAfterShutdown.test_processes_finished_gracefully[Strings] (known issue in: https://github.com/ros2/system_tests/issues/496 - CLOSED)
	  (also in: "nightly_win_rep" job) (known issue in: https://github.com/ros2/system_tests/issues/496 - CLOSED)
	  (also in: "nightly_win_rel" job) (known issue in: https://github.com/ros2/system_tests/issues/496 - CLOSED)

Issue #36 in nightly_linux-rhel_release
- Failing 6.25% in 1 out of 16 builds
	ros2bag.test.test_record_qos_profiles.test_record_qos_profiles
	  (also in: "nightly_linux_repeated" job)
	  (also in: "nightly_linux-rhel_release" job)
	  (also in: "nightly_linux-aarch64_repeated" job)

Issue #37 in nightly_linux_repeated
- Failing 6.25% in 1 out of 16 builds
	projectroot.test_publisher_subscriber__rclcpp__rmw_fastrtps_cpp__rmw_cyclonedds_cpp

Issue #38 in nightly_win_rel
- Failing 6.25% in 1 out of 16 builds
	test_rclcpp.TestNNodesAfterShutdown.test_10_nodes
	projectroot.test_n_nodes__rmw_fastrtps_cpp
	  (also in: "nightly_win_rep" job)
	  (also in: "nightly_linux-rhel_repeated" job)

Issue #39 in nightly_win_rep
- Failing 7.69% in 1 out of 13 builds
	rclcpp.TestServiceIntrospection.service_introspection_enable_disable_events
	  (also in: "nightly_win_rel" job)
	  (also in: "Rci__nightly-connext_ubuntu_noble_amd64" job)
	  (also in: "Jci__nightly-connext_ubuntu_noble_amd64" job)
	  (also in: "Ici__nightly-connext_ubuntu_jammy_amd64" job)

Issue #40 in nightly_win_rep
- Failing 7.69% in 1 out of 13 builds
	rclcpp.TestServiceIntrospection.service_introspection_enable_disable_event_content
	  (also in: "nightly_win_rel" job)
	  (also in: "Rci__nightly-connext_ubuntu_noble_amd64" job)
	  (also in: "Jci__nightly-connext_ubuntu_noble_amd64" job)
	  (also in: "Ici__nightly-connext_ubuntu_jammy_amd64" job)

Issue #41 in nightly_win_rep
- Failing 7.69% in 1 out of 13 builds
	rclcpp.TestService.server_qos
	  (also in: "nightly_win_rel" job)
	  (also in: "Rci__nightly-connext_ubuntu_noble_amd64" job)
	  (also in: "Jci__nightly-connext_ubuntu_noble_amd64" job)
	  (also in: "Ici__nightly-connext_ubuntu_jammy_amd64" job)

Issue #42 in nightly_win_rep
- Failing 7.69% in 1 out of 13 builds
	rclcpp.TestQosEvent.test_pub_matched_event_by_option_event_callback
	  (also in: "nightly_win_rel" job)
	  (also in: "Ici__nightly-connext_ubuntu_jammy_amd64" job)

Issue #43 in nightly_win_rep
- Failing 7.69% in 1 out of 13 builds
	rclcpp.TestQosEvent.test_on_new_event_callback
	rclcpp.TestQosEvent.test_invalid_on_new_event_callback
	  (also in: "nightly_win_rel" job)
	  (also in: "Hci__nightly-connext_ubuntu_jammy_amd64" job)

Issue #44 in nightly_win_rep
- Failing 7.69% in 1 out of 13 builds
	rclcpp.TestContentFilterSubscription.create_two_content_filters_with_same_topic_name_and_destroy
	  (also in: "nightly_win_rel" job)

Issue #45 in nightly_win_rep
- Failing 7.69% in 1 out of 13 builds
	rclcpp.TestSubscription.on_new_message_callback
	rclcpp.TestService.on_new_request_callback
	projectroot.test.rclcpp.test_subscription
	  (also in: "nightly_win_rel" job)
	  (also in: "Ici__nightly-connext_ubuntu_jammy_amd64" job)
	  (also in: "Hci__nightly-connext_ubuntu_jammy_amd64" job)

Issue #46 in nightly_win_rep
- Failing 7.69% in 1 out of 13 builds
	projectroot.test.rclcpp.test_service_introspection
	  (also in: "nightly_win_rel" job)
	  (also in: "Rci__nightly-connext_ubuntu_noble_amd64" job)
	  (also in: "Jci__nightly-connext_ubuntu_noble_amd64" job)
	  (also in: "Ici__nightly-connext_ubuntu_jammy_amd64" job)

Issue #47 in nightly_win_rep
- Failing 7.69% in 1 out of 13 builds
	projectroot.test.rclcpp.test_service
	  (also in: "nightly_win_rel" job)
	  (also in: "Rci__nightly-connext_ubuntu_noble_amd64" job)
	  (also in: "Jci__nightly-connext_ubuntu_noble_amd64" job)
	  (also in: "Ici__nightly-connext_ubuntu_jammy_amd64" job)
	  (also in: "Hci__nightly-connext_ubuntu_jammy_amd64" job)

Issue #48 in nightly_linux_repeated
- Failing 6.25% in 1 out of 16 builds
	rclcpp.TestNodeGraph.construct_from_node
	projectroot.test.rclcpp.test_node_interfaces__node_graph
	  (also in: "nightly_win_rep" job)
	  (also in: "nightly_win_rel" job)
	  (also in: "nightly_linux_repeated" job)

Issue #49 in nightly_win_rep
- Failing 7.69% in 1 out of 13 builds
	rclcpp.TestNodeClock.construct_from_node
	rclcpp.TestNodeBase.construct_from_node
	rclcpp.TestNode.wait_for_graph_event
	rclcpp.TestNode.undeclare_parameter
	rclcpp.TestNode.test_registering_multiple_callbacks_api
	rclcpp.TestNode.subnode_get_name_and_namespace
	rclcpp.TestNode.subnode_construction_and_destruction
	rclcpp.TestNode.static_and_dynamic_typing
	rclcpp.TestNode.set_parameters_undeclared_parameters_not_allowed
	rclcpp.TestNode.set_parameters_undeclared_parameters_allowed
	rclcpp.TestNode.set_parameters_atomically_undeclared_parameters_not_allowed
	rclcpp.TestNode.set_parameters_atomically_undeclared_parameters_allowed
	rclcpp.TestNode.set_parameter_undeclared_parameters_not_allowed
	rclcpp.TestNode.set_parameter_undeclared_parameters_allowed
	rclcpp.TestNode.parameter_uninitialized_exception_even_if_allow_undeclared
	rclcpp.TestNode.now
	rclcpp.TestNode.list_parameters
	rclcpp.TestNode.has_parameter
	rclcpp.TestNode.get_publishers_subscriptions_info_by_topic
	rclcpp.TestNode.get_parameters_undeclared_parameters_not_allowed
	rclcpp.TestNode.get_parameters_undeclared_parameters_allowed
	rclcpp.TestNode.get_parameter_with_node_allow_undeclared
	rclcpp.TestNode.get_parameter_undeclared_parameters_not_allowed
	rclcpp.TestNode.get_parameter_undeclared_parameters_allowed
	rclcpp.TestNode.get_parameter_types_undeclared_parameters_not_allowed
	rclcpp.TestNode.get_parameter_types_undeclared_parameters_allowed
	rclcpp.TestNode.get_parameter_or_with_return_value
	rclcpp.TestNode.get_parameter_or_undeclared_parameters_not_allowed
	rclcpp.TestNode.get_parameter_or_undeclared_parameters_allowed
	rclcpp.TestNode.get_name_and_namespace
	rclcpp.TestNode.get_logger
	rclcpp.TestNode.get_entity_names
	rclcpp.TestNode.get_clock
	rclcpp.TestNode.describe_parameters_undeclared_parameters_not_allowed
	rclcpp.TestNode.describe_parameters_undeclared_parameters_allowed
	rclcpp.TestNode.describe_parameter_undeclared_parameters_not_allowed
	rclcpp.TestNode.describe_parameter_undeclared_parameters_allowed
	rclcpp.TestNode.declare_parameters_with_no_initial_values
	rclcpp.TestNode.declare_parameter_with_vector
	rclcpp.TestNode.declare_parameter_with_overrides
	rclcpp.TestNode.declare_parameter_with_no_initial_values
	rclcpp.TestNode.declare_parameter_with_cli_overrides
	rclcpp.TestNode.declare_parameter_with_allow_undeclared_parameters
	rclcpp.TestNode.declare_parameter_allowed_simple_types_function_templates
	rclcpp.TestNode.declare_parameter_allowed_array_types_function_templates
	rclcpp.TestNode.create_sub_node_rmw_validate_namespace_error
	rclcpp.TestNode.construction_and_destruction
	rclcpp.TestNode.callback_groups
	projectroot.test.rclcpp.test_node_interfaces__node_clock
	projectroot.test.rclcpp.test_node_interfaces__node_base
	projectroot.test.rclcpp.test_node_interfaces__get_node_interfaces
	projectroot.test.rclcpp.test_node
	  (also in: "nightly_win_rel" job)

Issue #50 in nightly_win_rep
- Failing 7.69% in 1 out of 13 builds
	rclcpp.test_intra_process_within_one_node.type_adapted_messages_unique_pub_received_by_intra_process_subscription
	rclcpp.test_intra_process_within_one_node.type_adapted_messages_unique_ptr_ros_message_pub_received_by_intra_process_subscription
	rclcpp.test_intra_process_within_one_node.type_adapted_messages_ros_message_ref_pub_received_by_intra_process_subscription
	rclcpp.test_intra_process_within_one_node.type_adapted_messages_const_ref_pub_received_by_intra_process_subscription
	rclcpp.TestWaitForAllAckedWithParm/TestPublisherWaitForAllAcked.check_wait_for_all_acked_with_QosPolicy/2 ((88-byte object <01-00 00-00 00-00 00-00 01-00 00-00 00-00 00-00 01-00 00-00 02-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 
	rclcpp.TestWaitForAllAckedWithParm/TestPublisherWaitForAllAcked.check_wait_for_all_acked_with_QosPolicy/1 ((88-byte object <01-00 00-00 00-00 00-00 01-00 00-00 00-00 00-00 02-00 00-00 02-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 
	rclcpp.TestWaitForAllAckedWithParm/TestPublisherWaitForAllAcked.check_wait_for_all_acked_with_QosPolicy/0 ((88-byte object <01-00 00-00 00-00 00-00 01-00 00-00 00-00 00-00 01-00 00-00 02-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 
	rclcpp.TestTypeSupport.test_service_ts_set_params_srv
	rclcpp.TestTypeSupport.test_service_ts_set_params_atomically_srv
	rclcpp.TestTypeSupport.test_service_ts_list_params_srv
	rclcpp.TestTypeSupport.test_service_ts_get_params_srv_type
	rclcpp.TestTypeSupport.test_service_ts_get_params_srv
	rclcpp.TestTypeSupport.test_service_ts_get_parameters_types_srv
	rclcpp.TestTypeSupport.test_service_ts_describe_params_srv
	rclcpp.TestTypeSupport.basic_getters
	rclcpp.TestSubscriptionPublisherCount/one_context_test.increasing_and_decreasing_counts
	rclcpp.TestSubscription.various_creation_signatures
	rclcpp.TestSubscription.take_serialized
	rclcpp.TestSubscription.take
	rclcpp.TestSubscription.rcl_take_type_erased_error
	rclcpp.TestSubscription.rcl_take_serialized_message_error
	rclcpp.TestSubscription.rcl_subscription_init_error
	rclcpp.TestSubscription.rcl_subscription_get_publisher_count_error
	rclcpp.TestSubscription.rcl_subscription_get_actual_qos_error
	rclcpp.TestSubscription.rcl_subscription_fini_error
	rclcpp.TestSubscription.on_new_intra_process_message_callback
	rclcpp.TestSubscription.handle_loaned_message
	rclcpp.TestSubscription.get_network_flow_endpoints_errors
	rclcpp.TestSubscription.construction_and_destruction
	rclcpp.TestSubscription.check_type_adapted_messages_are_received_by_intra_process_subscription
	rclcpp.TestSubscription.check_type_adapted_messages_are_received_by_inter_process_subscription
	rclcpp.TestSubscription.callback_bind
	rclcpp.TestServiceIntrospection.service_introspection_nominal
	rclcpp.TestService.take_request
	rclcpp.TestService.server_qos_depth
	rclcpp.TestService.send_response
	rclcpp.TestService.rcl_service_response_publisher_get_actual_qos_error
	rclcpp.TestService.rcl_service_request_subscription_get_actual_qos_error
	rclcpp.TestService.construction_and_destruction_rcl_errors
	rclcpp.TestService.construction_and_destruction
	rclcpp.TestService.basic_public_getters
	rclcpp.TestSerializedMessageAllocator.borrow_from_subscription
	rclcpp.TestQosParameters.declare_with_id
	rclcpp.TestQosParameters.declare_with_callback
	rclcpp.TestQosParameters.declare_qos_subscription_parameters
	rclcpp.TestQosParameters.declare_no_parameters_interface
	rclcpp.TestQosParameters.declare
	rclcpp.TestQosEvent.test_subscription_constructor
	rclcpp.TestQosEvent.test_sub_matched_event_by_set_event_callback
	rclcpp.TestQosEvent.test_sub_matched_event_by_option_event_callback
	rclcpp.TestQosEvent.test_publisher_constructor
	rclcpp.TestQosEvent.test_pub_matched_event_by_set_event_callback
	rclcpp.TestQosEvent.test_default_incompatible_qos_callbacks
	rclcpp.TestQosEvent.execute
	rclcpp.TestQosEvent.construct_destruct_rcl_error
	rclcpp.TestQosEvent.add_to_wait_set
	rclcpp.TestPublisherThrows/TestPublisherInvalidIntraprocessQos.test_publisher_throws/keep_all_qos (keep_all_qos)                     
	rclcpp.TestPublisherSubscriptionCount/two_subscriptions_intraprocess_comm.increasing_and_decreasing_counts
	rclcpp.TestPublisherSub.construction_and_destruction
	rclcpp.TestPublisher.various_creation_signatures
	rclcpp.TestPublisher.test_large_message_unique
	rclcpp.TestPublisher.test_large_message_constref
	rclcpp.TestPublisher.serialized_message_publish
	rclcpp.TestPublisher.run_event_handlers
	rclcpp.TestPublisher.rcl_publisher_init_error
	rclcpp.TestPublisher.rcl_publisher_get_subscription_count_publisher_invalid
	rclcpp.TestPublisher.rcl_publisher_get_rmw_handle_error
	rclcpp.TestPublisher.rcl_publisher_get_options_error
	rclcpp.TestPublisher.rcl_publisher_get_gid_for_publisher_error
	rclcpp.TestPublisher.rcl_publisher_get_actual_qos_error
	rclcpp.TestPublisher.rcl_publisher_fini_error
	rclcpp.TestPublisher.publishers_equal_rmw_compare_gids_error
	rclcpp.TestPublisher.lowest_available_ipm_capacity
	rclcpp.TestPublisher.intra_process_transient_local
	rclcpp.TestPublisher.intra_process_publish_failures
	rclcpp.TestPublisher.inter_process_publish_failures
	rclcpp.TestPublisher.get_network_flow_endpoints_errors
	rclcpp.TestPublisher.do_loaned_message_publish_error
	rclcpp.TestPublisher.default_incompatible_qos_callback
	rclcpp.TestPublisher.conversion_exception_is_passed_up
	rclcpp.TestPublisher.construction_and_destruction
	rclcpp.TestPublisher.check_wait_for_all_acked_return
	rclcpp.TestPublisher.check_type_adapted_message_is_sent_and_received
	rclcpp.TestPublisher.basic_getters
	rclcpp.TestParameterService.set_parameters_atomically
	rclcpp.TestParameterService.set_parameters
	rclcpp.TestParameterService.list_parameters
	rclcpp.TestParameterService.get_parameters
	rclcpp.TestParameterService.get_parameter_types
	rclcpp.TestParameterService.describe_parameters
	rclcpp.TestParameterClient.sync_parameter_load_parameters
	rclcpp.TestParameterClient.sync_parameter_is_ready
	rclcpp.TestParameterClient.sync_parameter_get_parameters_allow_undeclared
	rclcpp.TestParameterClient.sync_parameter_get_parameters
	rclcpp.TestParameterClient.sync_parameter_get_parameter_types_allow_undeclared
	rclcpp.TestParameterClient.sync_parameter_get_parameter_types
	rclcpp.TestParameterClient.sync_parameter_get_parameter
	rclcpp.TestParameterClient.sync_parameter_event_subscription
	rclcpp.TestParameterClient.sync_parameter_describe_parameters_allow_undeclared
	rclcpp.TestParameterClient.sync_parameter_describe_parameters
	rclcpp.TestParameterClient.sync_parameter_delete_parameters
	rclcpp.TestParameterClient.sync_construction_and_destruction
	rclcpp.TestParameterClient.async_parameter_set_parameters_atomically
	rclcpp.TestParameterClient.async_parameter_load_parameters_from_map
	rclcpp.TestParameterClient.async_parameter_load_parameters_complicated_regex
	rclcpp.TestParameterClient.async_parameter_load_parameters
	rclcpp.TestParameterClient.async_parameter_load_no_valid_parameter
	rclcpp.TestParameterClient.async_parameter_load_from_map_no_valid_parameter
	rclcpp.TestParameterClient.async_parameter_list_parameters
	rclcpp.TestParameterClient.async_parameter_get_parameters_allow_undeclared
	rclcpp.TestParameterClient.async_parameter_get_parameters
	rclcpp.TestParameterClient.async_parameter_get_parameter_types_allow_undeclared
	rclcpp.TestParameterClient.async_parameter_get_parameter_types
	rclcpp.TestParameterClient.async_parameter_event_subscription
	rclcpp.TestParameterClient.async_parameter_describe_parameters_allow_undeclared
	rclcpp.TestParameterClient.async_parameter_describe_parameters
	rclcpp.TestParameterClient.async_parameter_delete_parameters
	rclcpp.TestParameterClient.async_construction_and_destruction
	rclcpp.TestNodeWithGlobalArgs.use_or_ignore_global_arguments
	rclcpp.TestNodeWithGlobalArgs.local_arguments_before_global
	rclcpp.TestNodeWaitables.add_remove_waitable
	rclcpp.TestNodeTypeDescriptions.interface_created
	rclcpp.TestNodeTopics.add_publisher
	rclcpp.TestNodeTimers.add_timer
	rclcpp.TestNodeService.add_service
	rclcpp.TestNodeParameters.construct_destruct_rcl_errors
	rclcpp.TestNodeInterfaces.node_interfaces_standard_interfaces
	rclcpp.TestNodeInterfaces.node_interfaces_nominal
	rclcpp.TestNodeInterfaces.ni_init
	rclcpp.TestNodeInterfaces.ni_all_init
	rclcpp.TestNodeInterfaces.default_constructor
	rclcpp.TestNode.SameParameterDifferentNode
	rclcpp.TestNode.RegisterParameterCallback
	rclcpp.TestNode.MultipleParameterCallbacks
	rclcpp.TestNode.LastInFirstCallForParameterEventCallbacks
	rclcpp.TestNode.LastInFirstCallForParameterCallbacks
	rclcpp.TestNode.GetParametersFromEvent
	rclcpp.TestNode.GetParameterFromEvent
	rclcpp.TestNode.EventCallback
	rclcpp.TestLogger.get_node_logger
	rclcpp.TestFindWeakNodes.allocator_strategy_with_weak_nodes
	rclcpp.TestFindWeakNodes.allocator_strategy_no_weak_nodes
	rclcpp.TestExternallyDefinedServices.extern_defined_uninitialized
	rclcpp.TestExternallyDefinedServices.extern_defined_initialized
	rclcpp.TestExternallyDefinedServices.extern_defined_destructor
	rclcpp.TestExternallyDefinedServices.default_behavior
	rclcpp.TestClockSleep.bad_clock_type
	rclcpp.RclcppGenericNodeFixture.publisher_and_subscriber_work
	rclcpp.RclcppGenericNodeFixture.publish_loaned_msg_work
	rclcpp.RclcppGenericNodeFixture.generic_subscription_uses_qos
	rclcpp.RclcppGenericNodeFixture.generic_subscription_different_callbacks
	rclcpp.RclcppGenericNodeFixture.generic_publisher_uses_qos
	projectroot.test.rclcpp.test_type_support
	projectroot.test.rclcpp.test_subscription_with_type_adapter
	projectroot.test.rclcpp.test_subscription_publisher_with_same_type_adapter
	projectroot.test.rclcpp.test_subscription_publisher_count_api
	projectroot.test.rclcpp.test_serialized_message_allocator
	projectroot.test.rclcpp.test_qos_parameters
	projectroot.test.rclcpp.test_qos_event__rmw_fastrtps_cpp
	projectroot.test.rclcpp.test_publisher_with_type_adapter
	projectroot.test.rclcpp.test_publisher_subscription_count_api (known issue in: https://github.com/ros2/rclcpp/issues/2230 - CLOSED)
	projectroot.test.rclcpp.test_publisher (known issue in: https://github.com/ros2/rclcpp/issues/2230 - CLOSED)
	projectroot.test.rclcpp.test_parameter_service
	projectroot.test.rclcpp.test_parameter_event_handler
	projectroot.test.rclcpp.test_parameter_client
	projectroot.test.rclcpp.test_node_interfaces__node_waitables
	projectroot.test.rclcpp.test_node_interfaces__node_type_descriptions
	projectroot.test.rclcpp.test_node_interfaces__node_topics
	projectroot.test.rclcpp.test_node_interfaces__node_timers
	projectroot.test.rclcpp.test_node_interfaces__node_services
	projectroot.test.rclcpp.test_node_interfaces__node_parameters
	projectroot.test.rclcpp.test_node_interfaces__node_interfaces
	projectroot.test.rclcpp.test_node_global_args
	projectroot.test.rclcpp.test_logger
	projectroot.test.rclcpp.test_generic_pubsub__rmw_fastrtps_cpp
	projectroot.test.rclcpp.test_find_weak_nodes
	projectroot.test.rclcpp.test_externally_defined_services
	  (also in: "nightly_win_rel" job)

Issue #51 in nightly_win_rep
- Failing 7.69% in 1 out of 13 builds
	test_rclcpp.test_time.timer_fire_regularly
	test_rclcpp.test_spin.spin_until_future_complete_interrupted
	rclpy.rclpy.test.test_client.TestClient.test_wait_for_service_5sec
	rclcpp.test_add_callback_groups_to_executor__rmw_fastrtps_cpp.gtest.missing_result
	rclcpp.TestMemoryStrategy.get_subscription_by_handle
	rclcpp.TestLoanedMessage.release
	rclcpp.TestLoanedMessage.move_loaned_message
	rclcpp.TestLoanedMessage.loan_from_pub
	rclcpp.TestLoanedMessage.initialize
	rclcpp.TestLoanedMessage.construct_with_loaned_message_publisher
	rclcpp.TestIntraProcessManagerWithAllocators.custom_allocator
	projectroot.test.rclcpp.test_loaned_message
	projectroot.test.rclcpp.test_intra_process_manager_with_allocators
	projectroot.test.rclcpp.test_add_callback_groups_to_executor__rmw_fastrtps_cpp
	projectroot.gtest_spin__rmw_fastrtps_cpp

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
buildfarmer-log Log of issues in the buildfarm
Projects
None yet
Development

No branches or pull requests

2 participants