unable to reproduce paper's results

[oferidan1]

Hi,

thanks alot for you great paper.
I am trying to run your pre-trained model over 7scenes.

when I try to run the original code I get below error:
RuntimeError: Error(s) in loading state_dict for PoseNetX_R2:
Unexpected key(s) in state_dict: "gnn2.mlp.0.weight", "gnn2.mlp.0.bias", "gnn2.mlp.2.weight", "gnn2.mlp.2.bias", "gnn2.mlp_updating.0.weight", "gnn2.mlp_updating.0.bias", "gnn2.mlp_updating.2.weight", "gnn2.mlp_updating.2.bias", "gnn2.edge_model.edge_mlp.0.weight", "gnn2.edge_model.edge_mlp.0.bias", "gnn2.edge_model.edge_mlp.2.weight", "gnn2.edge_model.edge_mlp.2.bias", "gnn2.att.g.weight", "gnn2.att.g.bias", "gnn2.att.theta.weight", "gnn2.att.theta.bias", "gnn2.att.phi.weight", "gnn2.att.phi.bias", "gnn2.att.W.weight", "gnn2.att.W.bias".

In order for the model to load properly, I changed in test.py line 162 from R2 to R3

from:
elif self.model_name == 'R3':
self.model = PoseNetX_R2(
to:
self.model = PoseNetX_R3(

and then model loads properly, but I am getting different results than in paper, for example for chess scene I get: 0.31m/14.18 degrees vs. paper 0.08m/2,7 degrees

python -u ${RELPOSEGNN}/python/niantic/testing/test.py
--dataset-dir "${SEVENSCENES}"
--test-data-dir "${SEVENSCENESRW}"
--weights "${DATADIR}/relpose_gnn__multi_39.pth.tar"
--save-dir "${DATADIR}"
--gpu 0
--test-scene "${SCENE}"

2023-09-09 18:16:05.220 | INFO | main:init:137 - Dataset: 7Scenes
2023-09-09 18:16:05.221 | INFO | main:init:138 - Test scene: chess
2023-09-09 18:16:05.221 | INFO | main:init:139 - Test data dir: relpose_gnn/7scenes-rw/
2023-09-09 18:16:05.221 | INFO | main:init:140 - Test dataset size: 2000
2023-09-09 18:16:05.221 | INFO | main:init:141 - Images sizes: 256, 341
2023-09-09 18:16:05.221 | INFO | main:init:142 - Number of nodes in the graph: 8, fc
2023-09-09 18:16:05.221 | INFO | main:init:143 - Number of nodes in the graph - test: 8 fc
2023-09-09 18:16:05.221 | INFO | main:init:145 - Use RP loss: True
2023-09-09 18:16:05.221 | INFO | main:init:146 - Use RP model: True
2023-09-09 18:16:05.221 | INFO | main:init:147 - srx: 0.0
2023-09-09 18:16:05.221 | INFO | main:init:148 - srq: -3
2023-09-09 18:16:05.221 | INFO | main:init:149 - edge_keep_factor: 0.5
2023-09-09 18:16:05.221 | INFO | main:init:150 - gnn_recursion: 2
2023-09-09 18:16:05.222 | INFO | main:init:151 - droprate: 0.5
2023-09-09 18:16:05.222 | INFO | main:init:152 - gpu: 0
2023-09-09 18:16:06.938 | INFO | main:init:175 - Num parameters: 118861132
2023-09-09 18:16:53.399 | INFO | main:eval_RP:275 - [Scene: chess, set: test, relpose_gnn__multi_39.pth.tar] Error in translation: median 0.31 m, mean 0.39 m Error in rotation: median 14.18 degrees, mean 16.58 degrees

any ideas?
what is the difference with PoseNetX_R2 and PoseNetX_R3?
are you sure you uploaded the correct pre-trainefd model to the git?

Thanks,
Ofer