the viewer in nerfstudio shows a rendered image from the 3DGS result at that current perspective，Is this right?

[smart4654154]

Thank you very much for your reply（https://github.com/nerfstudio-project/nerfstudio/issues/3090）
I am currently studying NERF and 3DGS, and I have some new question

1.What does "at that current perspective" mean, and how does nerfstudio obtain current perspective ?
2.the viewer shows a rendered image from the 3DGS result at that current perspective，Is this right?
thank you

[brentyi]

Hi!

1. "At the current perspective" means using a 3D camera defined by the web-based viewer. We use something similar to the "OrbitControls" controller in threejs to interact with the viewing camera pose: https://threejs.org/examples/misc_controls_orbit.html
2. Yes, cameras for NeRFs and 3DGS are handled identically in the context of the viewer!

[smart4654154]

Thank you very much for your reply
I would like to know more about this content. Can you give me some suggestions?
When I rotate in the viewer, the Python code will get the current perspective. The Python code uses the perspective information to re-render the image and pass it to the viewer. I would like to know the Python code for this process. Can you give me some suggestions?
Thank you very much.

[brentyi]

There's a lot of steps here so it's hard to write a short summary, but you could look at:

• The communication infra: https://github.com/nerfstudio-project/viser/tree/main/src/viser/infra
• CameraHandle in viser: https://github.com/search?q=repo%3Anerfstudio-project%2Fviser%20CameraHandle&type=code
• The nerfstudio viewer code: https://github.com/nerfstudio-project/nerfstudio/blob/main/nerfstudio/viewer/viewer.py

Nerfview might also be a useful reference: https://github.com/hangg7/nerfview