Trajectory error when running custom dataset

[shizhinian]

Hello, I want to use monocular mode to run other datasets. I chose this data set. I have converted the data set into the same format as Euroc, and modified ImuParams.yaml and LeftCameraParams.yaml according to the parameters provided by the dataset. the program can indeed run, but the running results are very bad, the trajectory has very serious divergence and drift. The following is some information printed by the terminal during the running process:

visualizer_frontend_queue Size [#]                     140      16.5    {2.7 +- 2.3}    [1.0, 7.0]      1.0
W0923 13:10:42.549254 98486 OpenCvVisualizer3D.cpp:1202] No landmark information for Visualizer. Not displaying 3D points.
W0923 13:10:42.636262 98486 OpenCvVisualizer3D.cpp:1202] No landmark information for Visualizer. Not displaying 3D points.
W0923 13:10:42.703012 98486 OpenCvVisualizer3D.cpp:1202] No landmark information for Visualizer. Not displaying 3D points.
W0923 13:10:42.762836 98486 OpenCvVisualizer3D.cpp:1202] No landmark information for Visualizer. Not displaying 3D points.
W0923 13:10:42.828842 98486 OpenCvVisualizer3D.cpp:1202] No landmark information for Visualizer. Not displaying 3D points.
W0923 13:10:42.843916 98485 PipelineModule.h:338] Callbacks for module: VioBackend are taking very long! Current latency: 15 ms.
W0923 13:10:42.885584 98486 OpenCvVisualizer3D.cpp:1202] No landmark information for Visualizer. Not displaying 3D points.
W0923 13:10:42.939684 98486 OpenCvVisualizer3D.cpp:1202] No landmark information for Visualizer. Not displaying 3D points.
W0923 13:10:43.021660 98486 OpenCvVisualizer3D.cpp:1202] No landmark information for Visualizer. Not displaying 3D points.
I0923 13:10:43.036077 98489 Pipeline.cpp:180] Statistics
-----------                                           # log hz  {avg +- std}    [min, max]      last
Display [ms]                                            66      7.4     {132.6 +- 56.6} [70.0, 468.0]   170.0
VioBackend [ms]                                        147      16.3    {41.6 +- 12.4}  [1.0, 133.0]    44.0
VioFrontend Frame Rate [ms]                            208      23.4    {10.3 +- 7.1}   [3.0, 31.0]     4.0
VioFrontend Keyframe Rate [ms]                         147      16.6    {43.5 +- 8.8}   [21.0, 92.0]    33.0
VioFrontend [ms]                                       356      39.5    {24.4 +- 16.9}  [3.0, 92.0]     5.0
Visualizer [ms]                                        146      16.3    {35.2 +- 10.2}  [4.0, 74.0]     40.0
backend_input_queue Size [#]                           148      16.4    {2.2 +- 1.3}    [1.0, 6.0]      1.0
data_provider_left_frame_queue Size [#]                946      104.7   {275.9 +- 18.4} [1.0, 582.0]    582.0
display_input_queue Size [#]                           293      32.5    {117.1 +- 21.1} [1.0, 226.0]    226.0
frontend_input_queue Size [#]                          362      40.1    {5.0 +- 0.1}    [1.0, 5.0]      5.0
visualizer_backend_queue Size [#]                      147      16.3    {1.0 +- 0.1}    [1.0, 2.0]      1.0
visualizer_frontend_queue Size [#]                     148      16.4    {2.6 +- 1.9}    [1.0, 7.0]      2.0
W0923 13:10:43.067785 98486 OpenCvVisualizer3D.cpp:1202] No landmark information for Visualizer. Not displaying 3D points.
W0923 13:10:43.123886 98486 OpenCvVisualizer3D.cpp:1202] No landmark information for Visualizer. Not displaying 3D points.
W0923 13:10:43.186532 98486 OpenCvVisualizer3D.cpp:1202] No landmark information for Visualizer. Not displaying 3D points.
W0923 13:10:43.233086 98486 OpenCvVisualizer3D.cpp:1202] No landmark information for Visualizer. Not displaying 3D points.
W0923 13:10:43.308744 98486 OpenCvVisualizer3D.cpp:1202] No landmark information for Visualizer. Not displaying 3D points.
I0923 13:10:43.361608 98484 Tracker.cpp:367] Low mono disparity.
W0923 13:10:43.374199 98486 OpenCvVisualizer3D.cpp:1202] No landmark information for Visualizer. Not displaying 3D points.
W0923 13:10:43.401669 98485 VioBackend.cpp:366] Low disparity: adding zero velocity and no motion factors.
W0923 13:10:43.422362 98486 OpenCvVisualizer3D.cpp:1202] No landmark information for Visualizer. Not displaying 3D points.
I0923 13:10:43.515455 98484 Tracker.cpp:367] Low mono disparity.
W0923 13:10:43.540387 98485 PipelineModule.h:338] Callbacks for module: VioBackend are taking very long! Current latency: 13 ms.
I0923 13:10:43.536330 98489 Pipeline.cpp:180] Statistics
-----------                                           # log hz  {avg +- std}    [min, max]      last
Display [ms]                                            70      7.4     {132.4 +- 55.4} [70.0, 468.0]   126.0
VioBackend [ms]                                        155      16.3    {41.6 +- 12.1}  [1.0, 133.0]    75.0
VioFrontend Frame Rate [ms]                            221      23.6    {10.4 +- 7.8}   [3.0, 35.0]     4.0
VioFrontend Keyframe Rate [ms]                         154      16.5    {43.4 +- 8.5}   [21.0, 92.0]    44.0
VioFrontend [ms]                                       376      39.6    {24.4 +- 17.1}  [3.0, 92.0]     4.0
Visualizer [ms]                                        154      16.3    {35.6 +- 9.5}   [4.0, 74.0]     38.0
backend_input_queue Size [#]                           155      16.4    {2.1 +- 0.9}    [1.0, 6.0]      1.0
data_provider_left_frame_queue Size [#]                994      104.2   {291.3 +- 17.0} [1.0, 610.0]    610.0
display_input_queue Size [#]                           308      32.5    {122.8 +- 21.3} [1.0, 238.0]    238.0
frontend_input_queue Size [#]                          382      40.2    {5.0 +- 0.1}    [1.0, 5.0]      5.0
visualizer_backend_queue Size [#]                      155      16.3    {1.0 +- 0.1}    [1.0, 2.0]      1.0
visualizer_frontend_queue Size [#]                     155      16.4    {2.5 +- 1.4}    [1.0, 7.0]      1.0
W0923 13:10:43.554668 98485 VioBackend.cpp:366] Low disparity: adding zero velocity and no motion factors.
I0923 13:10:43.576340 98484 Tracker.cpp:367] Low mono disparity.
W0923 13:10:43.593755 98486 OpenCvVisualizer3D.cpp:1202] No landmark information for Visualizer. Not displaying 3D points.
W0923 13:10:43.618026 98485 VioBackend.cpp:366] Low disparity: adding zero velocity and no motion factors.
I0923 13:10:43.642572 98484 Tracker.cpp:367] Low mono disparity.
W0923 13:10:43.646327 98486 OpenCvVisualizer3D.cpp:1202] No landmark information for Visualizer. Not displaying 3D points.
W0923 13:10:43.684006 98485 VioBackend.cpp:366] Low disparity: adding zero velocity and no motion factors.
W0923 13:10:43.718164 98486 OpenCvVisualizer3D.cpp:1202] No landmark information for Visualizer. Not displaying 3D points.
I0923 13:10:43.727535 98484 Tracker.cpp:367] Low mono disparity.
W0923 13:10:43.768707 98485 VioBackend.cpp:366] Low disparity: adding zero velocity and no motion factors.
I0923 13:10:43.787870 98484 Tracker.cpp:367] Low mono disparity.
W0923 13:10:43.789732 98486 OpenCvVisualizer3D.cpp:1202] No landmark information for Visualizer. Not displaying 3D points.
W0923 13:10:43.839053 98485 VioBackend.cpp:366] Low disparity: adding zero velocity and no motion factors.
W0923 13:10:43.864022 98486 OpenCvVisualizer3D.cpp:1202] No landmark information for Visualizer. Not displaying 3D points.
W0923 13:10:43.884450 98485 PipelineModule.h:338] Callbacks for module: VioBackend are taking very long! Current latency: 20 ms.
E0923 13:10:43.924181 98485 VioBackend.cpp:1409] 
Indeterminant linear system detected while working near variable
8646911284551352459 (Symbol: x139).

Thrown when a linear system is ill-posed.  The most common cause for this
error is having underconstrained variables.  Mathematically, the system is
underdetermined.  See the GTSAM Doxygen documentation at
http://borg.cc.gatech.edu/ on gtsam::IndeterminantLinearSystemException for
more information.
E0923 13:10:43.924646 98485 VioBackend.cpp:1412] ERROR: Variable has type 'x' and index 139
E0923 13:10:43.925149 98485 VioBackend.cpp:1440] Adding prior on key: x139
E0923 13:10:43.925276 98485 VioBackend.cpp:1440] Adding prior on key: x134
E0923 13:10:43.925302 98485 VioBackend.cpp:1440] Adding prior on key: b139
E0923 13:10:43.925357 98485 VioBackend.cpp:1440] Adding prior on key: b134
E0923 13:10:43.925392 98485 VioBackend.cpp:1440] Adding prior on key: v139
E0923 13:10:43.925453 98485 VioBackend.cpp:1440] Adding prior on key: v134
E0923 13:10:43.925561 98485 VioBackend.cpp:1488] Attempting to update smoother with added prior factors
E0923 13:10:43.956192 98485 VioBackend.cpp:1494] Smoother recovery failed. Most likely, the additional prior factors were insufficient to keep the system from becoming indeterminant.
E0923 13:10:43.957327 98485 VioBackendModule.cpp:34] Backend did not return an output: shutting down Backend.
W0923 13:10:43.983662 98484 FeatureDetector.cpp:160] No corners extracted for frame with id: 394
W0923 13:10:44.024293 98484 FeatureDetector.cpp:160] No corners extracted for frame with id: 397
I0923 13:10:44.040876 98489 Pipeline.cpp:180] Statistics
-----------                                           # log hz  {avg +- std}    [min, max]      last
Display [ms]                                            74      7.4     {132.1 +- 54.7} [70.0, 468.0]   99.0
VioBackend [ms]                                        161      16.2    {41.4 +- 11.8}  [1.0, 133.0]    59.0
VioFrontend Frame Rate [ms]                            234      23.7    {10.4 +- 7.7}   [3.0, 35.0]     13.0
VioFrontend Keyframe Rate [ms]                         163      16.5    {43.4 +- 8.5}   [21.0, 92.0]    26.0
VioFrontend [ms]                                       397      39.7    {24.4 +- 17.5}  [3.0, 92.0]     26.0
Visualizer [ms]                                        160      16.2    {36.0 +- 9.1}   [4.0, 74.0]     44.0
backend_input_queue Size [#]                           162      16.3    {2.1 +- 0.3}    [1.0, 6.0]      1.0
data_provider_left_frame_queue Size [#]               1053      104.9   {310.3 +- 18.6} [1.0, 648.0]    648.0
display_input_queue Size [#]                           323      32.2    {128.3 +- 21.4} [1.0, 248.0]    248.0
frontend_input_queue Size [#]                          403      40.2    {5.0 +- 0.1}    [1.0, 5.0]      5.0
visualizer_backend_queue Size [#]                      160      16.2    {1.0 +- 0.1}    [1.0, 2.0]      1.0
visualizer_frontend_queue Size [#]                     164      16.4    {2.5 +- 0.5}    [1.0, 7.0]      4.0
W0923 13:10:44.065312 98484 FeatureDetector.cpp:160] No corners extracted for frame with id: 399
W0923 13:10:44.133656 98484 FeatureDetector.cpp:160] No corners extracted for frame with id: 402
W0923 13:10:44.209336 98484 FeatureDetector.cpp:160] No corners extracted for frame with id: 405
W0923 13:10:44.249686 98484 FeatureDetector.cpp:160] No corners extracted for frame with id: 407
W0923 13:10:44.308528 98484 FeatureDetector.cpp:160] No corners extracted for frame with id: 410
W0923 13:10:44.340198 98484 FeatureDetector.cpp:160] No corners extracted for frame with id: 412
W0923 13:10:44.374863 98484 FeatureDetector.cpp:160] No corners extracted for frame with id: 414
W0923 13:10:44.417986 98484 FeatureDetector.cpp:160] No corners extracted for frame with id: 416
W0923 13:10:44.485502 98484 FeatureDetector.cpp:160] No corners extracted for frame with id: 420
W0923 13:10:44.529414 98484 FeatureDetector.cpp:160] No corners extracted for frame with id: 423
I0923 13:10:44.541154 98489 Pipeline.cpp:180] Statistics
-----------                                           # log hz  {avg +- std}    [min, max]      last
Display [ms]                                            79      7.5     {130.2 +- 53.6} [70.0, 468.0]   113.0
VioBackend [ms]                                        161      16.2    {41.4 +- 11.8}  [1.0, 133.0]    59.0
VioFrontend Frame Rate [ms]                            249      24.0    {10.4 +- 7.7}   [3.0, 35.0]     8.0
VioFrontend Keyframe Rate [ms]                         174      16.8    {42.4 +- 9.9}   [21.0, 92.0]    25.0
VioFrontend [ms]                                       424      40.3    {24.0 +- 16.5}  [3.0, 92.0]     8.0
Visualizer [ms]                                        160      16.2    {36.0 +- 9.1}   [4.0, 74.0]     44.0
backend_input_queue Size [#]                           162      16.3    {2.1 +- 0.3}    [1.0, 6.0]      1.0
data_provider_left_frame_queue Size [#]               1126      106.9   {333.7 +- 18.6} [1.0, 695.0]    695.0
display_input_queue Size [#]                           334      31.7    {132.4 +- 21.2} [1.0, 254.0]    254.0
frontend_input_queue Size [#]                          430      40.8    {5.0 +- 0.0}    [1.0, 5.0]      5.0
visualizer_backend_queue Size [#]                      160      16.2    {1.0 +- 0.1}    [1.0, 2.0]      1.0
visualizer_frontend_queue Size [#]                     175      16.6    {3.0 +- 3.0}    [1.0, 15.0]     15.0
W0923 13:10:44.565397 98484 FeatureDetector.cpp:160] No corners extracted for frame with id: 425
W0923 13:10:44.601959 98484 FeatureDetector.cpp:160] No corners extracted for frame with id: 427
W0923 13:10:44.635910 98484 FeatureDetector.cpp:160] No corners extracted for frame with id: 429
W0923 13:10:44.673436 98484 FeatureDetector.cpp:160] No corners extracted for frame with id: 432
W0923 13:10:44.709687 98484 FeatureDetector.cpp:160] No corners extracted for frame with id: 434
W0923 13:10:44.755023 98484 FeatureDetector.cpp:160] No corners extracted for frame with id: 437
W0923 13:10:44.789541 98484 FeatureDetector.cpp:160] No corners extracted for frame with id: 439
W0923 13:10:44.817885 98484 FeatureDetector.cpp:160] No corners extracted for frame with id: 441
W0923 13:10:44.851533 98484 FeatureDetector.cpp:160] No corners extracted for frame with id: 443
I0923 13:10:45.041515 98489 Pipeline.cpp:180] Statistics
-----------                                           # log hz  {avg +- std}    [min, max]      last
Display [ms]                                            85      7.7     {127.0 +- 53.1} [67.0, 468.0]   67.0
VioBackend [ms]                                        161      16.2    {41.4 +- 11.8}  [1.0, 133.0]    59.0
VioFrontend Frame Rate [ms]                            266      24.5    {10.2 +- 7.5}   [3.0, 35.0]     5.0
VioFrontend Keyframe Rate [ms]                         187      17.2    {41.4 +- 10.9}  [21.0, 92.0]    30.0
VioFrontend [ms]                                       454      41.2    {23.5 +- 14.6}  [3.0, 92.0]     5.0
Visualizer [ms]                                        160      16.2    {36.0 +- 9.1}   [4.0, 74.0]     44.0
backend_input_queue Size [#]                           162      16.3    {2.1 +- 0.3}    [1.0, 6.0]      1.0
data_provider_left_frame_queue Size [#]               1203      109.0   {358.3 +- 17.3} [1.0, 741.0]    741.0
display_input_queue Size [#]                           347      31.5    {137.1 +- 20.6} [1.0, 261.0]    261.0
frontend_input_queue Size [#]                          460      41.7    {5.0 +- 0.0}    [1.0, 5.0]      5.0
visualizer_backend_queue Size [#]                      160      16.2    {1.0 +- 0.1}    [1.0, 2.0]      1.0
visualizer_frontend_queue Size [#]                     188      17.1    {4.3 +- 7.4}    [1.0, 28.0]     28.0
W0923 13:10:45.058884 98484 FeatureDetector.cpp:160] No corners extracted for frame with id: 455
W0923 13:10:45.092196 98484 FeatureDetector.cpp:160] No corners extracted for frame with id: 457
W0923 13:10:45.153076 98484 FeatureDetector.cpp:160] No corners extracted for frame with id: 460
W0923 13:10:45.183002 98484 FeatureDetector.cpp:160] No corners extracted for frame with id: 462
W0923 13:10:45.223311 98484 FeatureDetector.cpp:160] No corners extracted for frame with id: 464
W0923 13:10:45.258793 98484 FeatureDetector.cpp:160] No corners extracted for frame with id: 466
W0923 13:10:45.300591 98484 FeatureDetector.cpp:160] No corners extracted for frame with id: 469
W0923 13:10:45.335089 98484 FeatureDetector.cpp:160] No corners extracted for frame with id: 471
W0923 13:10:45.394138 98484 FeatureDetector.cpp:160] No corners extracted for frame with id: 474
W0923 13:10:45.433764 98484 FeatureDetector.cpp:160] No corners extracted for frame with id: 476
W0923 13:10:45.530064 98484 FeatureDetector.cpp:160] No corners extracted for frame with id: 481
I0923 13:10:45.541965 98489 Pipeline.cpp:180] Statistics
-----------                                           # log hz  {avg +- std}    [min, max]      last
Display [ms]                                            89      7.8     {126.1 +- 52.3} [67.0, 468.0]   109.0
VioBackend [ms]                                        161      16.2    {41.4 +- 11.8}  [1.0, 133.0]    59.0
VioFrontend Frame Rate [ms]                            282      24.8    {10.1 +- 7.5}   [3.0, 35.0]     7.0
VioFrontend Keyframe Rate [ms]                         199      17.5    {40.6 +- 10.2}  [21.0, 92.0]    34.0
VioFrontend [ms]                                       482      41.8    {23.1 +- 11.9}  [3.0, 92.0]     7.0
Visualizer [ms]                                        160      16.2    {36.0 +- 9.1}   [4.0, 74.0]     44.0
backend_input_queue Size [#]                           162      16.3    {2.1 +- 0.3}    [1.0, 6.0]      1.0
data_provider_left_frame_queue Size [#]               1264      109.6   {377.7 +- 16.9} [1.0, 775.0]    775.0
display_input_queue Size [#]                           359      31.1    {141.4 +- 20.0} [1.0, 269.0]    269.0
frontend_input_queue Size [#]                          488      42.3    {5.0 +- 0.0}    [1.0, 5.0]      5.0
visualizer_backend_queue Size [#]                      160      16.2    {1.0 +- 0.1}    [1.0, 2.0]      1.0
visualizer_frontend_queue Size [#]                     200      17.3    {6.1 +- 12.1}   [1.0, 40.0]     40.0
W0923 13:10:45.571712 98484 FeatureDetector.cpp:160] No corners extracted for frame with id: 483
W0923 13:10:45.611068 98484 FeatureDetector.cpp:160] No corners extracted for frame with id: 486
W0923 13:10:45.648015 98484 FeatureDetector.cpp:160] No corners extracted for frame with id: 488
W0923 13:10:45.693670 98484 FeatureDetector.cpp:160] No corners extracted for frame with id: 491
W0923 13:10:45.736593 98484 FeatureDetector.cpp:160] No corners extracted for frame with id: 493
W0923 13:10:45.781719 98484 FeatureDetector.cpp:160] No corners extracted for frame with id: 495
W0923 13:10:45.830278 98484 FeatureDetector.cpp:160] No corners extracted for frame with id: 498
W0923 13:10:45.868050 98484 FeatureDetector.cpp:160] No corners extracted for frame with id: 500
W0923 13:10:45.913311 98484 FeatureDetector.cpp:160] No corners extracted for frame with id: 503
W0923 13:10:45.947100 98484 FeatureDetector.cpp:160] No corners extracted for frame with id: 505
I0923 13:10:46.042515 98489 Pipeline.cpp:180] Statistics
-----------                                           # log hz  {avg +- std}    [min, max]      last
Display [ms]                                            93      7.8     {126.1 +- 52.0} [67.0, 468.0]   83.0
VioBackend [ms]                                        161      16.2    {41.4 +- 11.8}  [1.0, 133.0]    59.0
VioFrontend Frame Rate [ms]                            298      25.1    {10.0 +- 6.0}   [3.0, 35.0]     5.0
VioFrontend Keyframe Rate [ms]                         211      17.8    {40.0 +- 10.2}  [21.0, 92.0]    31.0
VioFrontend [ms]                                       510      42.4    {22.8 +- 10.9}  [3.0, 92.0]     5.0
Visualizer [ms]                                        160      16.2    {36.0 +- 9.1}   [4.0, 74.0]     44.0
backend_input_queue Size [#]                           162      16.3    {2.1 +- 0.3}    [1.0, 6.0]      1.0
data_provider_left_frame_queue Size [#]               1336      111.0   {400.2 +- 16.9} [1.0, 818.0]    818.0
display_input_queue Size [#]                           371      30.8    {145.6 +- 19.3} [1.0, 277.0]    277.0
frontend_input_queue Size [#]                          516      42.9    {5.0 +- 0.0}    [1.0, 5.0]      5.0
visualizer_backend_queue Size [#]                      160      16.2    {1.0 +- 0.1}    [1.0, 2.0]      1.0
visualizer_frontend_queue Size [#]                     212      17.6    {8.4 +- 16.8}   [1.0, 52.0]     52.0

Here is a video of the program in action.
My question is:

• Do you know what causes the drift?
• Could it be one of the reasons for the message No landmark information for Visualizer. Not displaying 3D points. or No corners extracted for frame with id: xxx prompted in the terminal?
• How should I troubleshoot the error?

Looking forward to your answer.
Please give also the following information:

• Kimera-VIO: current master 641576
• GTSAM: borglab/ gtsam@37ea955
• OpenGV: current master laurentkneip/ opengv@91f4b19
• OpenCV: 4.2.1
• OS: Ubuntu 20.04
• Did you change the source code? (yes / no): no