Use of AprilTag ROS2 with Realsense D405 cameras?

[tdivic-muddy]

I am trying to use this AprilTag ROS2 software package for AprilTag detection using the Intel RealSense D405 camera. I've successfully set up the code to detect the AprilTags on the image from topic /camera0/color/image_rect_raw, and on the 2D image the origin of the tag is correctly identified. However, when I look at the TF of the tag in 3D space, it is clearly not in the right location.

I've double checked that I've got the right tag size set in my parameters, but can't seem to resolve my issue.

In general, I'm surprised by how few examples/documentation I've seen for anyone else using AprilTags in ROS2 with the Realsense cameras. Is anyone else doing, and if so, did you have success?

[MartyG-RealSense]

Hi @tdivic-muddy It is more common to use the Aruco type of fiducial image tag with RealSense 400 Series cameras instead of Apriltags (which have been used with the now-retired L515 and T265 models of RealSense camera).

The link below has a ROS2 version of apriltag_ros, though it is not designed for RealSense specifically.

https://github.com/christianrauch/apriltag_ros

A RealSense-tested project is apriltag_docking, though it was designed for ROS2 Foxy rather than Humble.

https://github.com/Adlink-ROS/apriltag_docking

---

In regard to Aruco tags, aruco_ros supports ROS2 Humble (again, it is not RealSense-specific).

https://github.com/pal-robotics/aruco_ros

ros2_aruco_pose_estimation has been tested with RealSense D435 though.

https://github.com/AIRLab-POLIMI/ros2-aruco-pose-estimation

[tdivic-muddy]

Is there a reason RealSense users more commonly use the Aruco fiducials? From my understanding, either Apriltags or Aruco should be fine.

Thanks for the links. The first one you linked is the same one I linked that I'm currently using, but I'll take a look at the second project for more info.

In general I've got the apriltag_ros library to work, but the 3D pose estimate is off. All tags seem to be projected along the camera-face normal vector much further than they should be.

[MartyG-RealSense]

I am not aware of a reason why Apriltags cannot be used with RealSense 400 Series cameras. Aruco was simply always the most popular option, perhaps because people tend to choose what others before them have already successfully used.

The D405 camera model does not have a dedicated RGB sensor and the RGB image is instead provided by the depth sensor and passed through an Image Signal Processor (ISP) chip to generate the final RGB image. So unlike other 400 Series cameras where the RGB viewpoint is horizontally offset from the depth viewpoint (because the physical RGB sensor on the front of the camera is offset in position from the physical left IR sensor), depth and RGB images on D405 have the same viewpoint because they share the same sensor.