Keep WAITING FOR IMAGES with black screen

[Vench1]

I tried to test orb-slam3 using ros2 run orbslam3 mono /home/zjp/ORB/ORB_SLAM3/Vocabulary/ORBvoc.txt /home/zjp/ORB/ORB_SLAM3/Examples/Monocular/TUM1.yaml.
I modified the image topic in ORB_SLAM3_ROS2/src/monocular/monocular-slam-node.cpp in advance to the topic published by my oak-d camera.
I also checked it with rqt_graph, but it still shows a black screen and WAITING FOR IMAGES. It is also blank in the map viewer.

I saw someone say it is because of the camera resolution, but I actually only use 320×180 images. I checked the bandwidth and it is only 2M. Someone else said that you can try it a little longer. He mentioned that he went to the bathroom to wash his hands and came back to find that it was running normally (here). Now, my hands are as clean as they can be, but I am still WAITING FOR IMAGES.

Does anyone have any thoughts on this issue?

[dirksavage88]

I have the same issue with one frame has been sent but no display.

Are you using a real camera? I have tried resizing my rpi cam v2 down to 640 x 480 but still get the same result even though I am waiting and it does detect a frame being sent. So this leads me to believe something is wrong with tracking:

ORB_SLAM3_ROS2/src/monocular/monocular-slam-node.cpp

Line 42 in 00c5433

m_SLAM->TrackMonocular(m_cvImPtr->image, Utility::StampToSec(msg->header.stamp));

However as soon as I switch from real camera to a ros2 bag file (Viconeasy 101 easy for example) it tracks fine and I see the images + tracking path

[mruck03]

Having the same issue. Any breakthroughs?

[Vench1]

I have the same issue with one frame has been sent but no display.

Are you using a real camera? I have tried resizing my rpi cam v2 down to 640 x 480 but still get the same result even though I am waiting and it does detect a frame being sent. So this leads me to believe something is wrong with tracking:

ORB_SLAM3_ROS2/src/monocular/monocular-slam-node.cpp

Line 42 in 00c5433

m_SLAM->TrackMonocular(m_cvImPtr->image, Utility::StampToSec(msg->header.stamp));

However as soon as I switch from real camera to a ros2 bag file (Viconeasy 101 easy for example) it tracks fine and I see the images + tracking path

Hi, @dirksavage88. Yes, I'm using a real camera. I think 640 x 480 is adequate as well. Recently I realized that I just have to wait about 15-25 minutes and the picture will appear, but it is unbearable.

I found another solution:

1. First, we will use the default camera parameters to do a test, so you need to find the path of your default camera parameter file (e.g. ORB_SLAM3/Examples/Monocular/TUM1. yaml), I've found that sometimes my own camera intrinsics won't work, but rest assured that modifying the resolution is still no problem.
2. then make sure that there are some feature-rich objects in front of where the robot or the camera is shooting (not against a white wall),
3. and last but not least, run ros2 run orbslam3 mono /your/path/to/ORBvoc.txt /your/path/to/TUM1. yaml After that, you need to go and move your robot or camera around a bit (translate and rotate a certain distance). Surprisingly, it's very quick to get a picture (10 seconds or so).

I think it's orb-slam3 that needs the camera to have a certain translation when it's initialized, although this seems to be often used to initialize imu, but this worked for me. So you might as well try moving your robot or camera at the beginning when using a real camera.

[Vench1]

Having the same issue. Any breakthroughs?

Hi, @mruck03 . After running your code, you can try to move your robot or camera to make it have a certain translation and rotation. This seems to make orb-slam3 initialize quickly. When I tried it, I found that it can run normally in about 10 seconds. For details, please refer to the reply above. I hope it helps you.

[dirksavage88]

@Vench1 thanks, it was just a matter of moving the camera and then waiting patiently. It looks like even with an rpi cam it can work! Definitely doesn't do well with motion blur and any blank walls, but those are known limitations

[andremaurell]

Hey guys nice thread, but, can anyone help-me? when I run the mono, just stuck on "starting the viewer", and didn't show "one frame has been sent" like yours, I'm trying to use gazebo

[mruck03]

Hey, make sure that you are subscribing to the correct ros topic in monocular-slam-node.cpp, Then if you move your camera around in gazebo it should start working if there are enough features to detect within the cameras vision. If not, then it will stay black until it can actually detect features within the image.

[andremaurell]

like that?