Geometry Plugin

Since geometric deep learning is rising, there is the need of a tensorboard plugin to visualize geometric data. In comparison to the Mesh plugin from tensorboard, this plugin is more stable and offers the opportunity to add feature vectors for each vertex.

Main features:

• method to extend the SummaryWriter
• visualize point clouds, meshes, and 3D feature vectors for each vertex
• apply colormaps from colormap to point clouds and 3D feature vectors
• perspective and orthografic visualization
• save the visualization as .png
• normalize scale of geometries and features

Known issues:

• large geometries (above 40K vertices) with feature vectors need some time to be loaded and visualized
• faces cannot be colored separately. Either use point clouds or color the whole mesh (see Usage).

Future features:

• Use official tensorboard api, when finally provided.
• If you have requests, please create an issue.

Installation

$ pip install tensorboard-plugin-geometry

Usage

Currently, only pytorch is supported. Since it offeres a great library geometric pytorch to apply machine learning to graphs.

write summaries

To write summaries, load the summary writing method and add this to the SummaryWriter.

from torch.utils.tensorboard import SummaryWriter

from tensorboard_plugin_geometry import add_geometry

# add writer function from this package
SummaryWriter.add_geometry = add_geometry

writer = SummaryWriter(log_dir='/a/path/to/logs')

# write data
writer.add_geometry('a beautiful tag', pos.reshape(1, 100, 3), features=wss.reshape(1, 100, 3), global_step=1)

Tip for tensorboard

If this dashboard should be use for visualizing final results, the option samples_per_plugin might be of interest:

--samples_per_plugin SAMPLES_PER_PLUGIN
                      An optional comma separated list of
                      plugin_name=num_samples pairs to explicitly specify
                      how many samples to keep per tag for that plugin. For
                      unspecified plugins, TensorBoard randomly downsamples
                      logged summaries to reasonable values to prevent out-
                      of-memory errors for long running jobs. This flag
                      allows fine control over that downsampling. Note that
                      0 means keep all samples of that type. For instance
                      "scalars=500,images=0" keeps 500 scalars and all
                      images. Most users should not need to set this flag.

For this plugin, tensors are of interest:

tensorboard --logdir=./logs --samples_per_plugin tensors=0 --tag=groundtruth

Docs

add_geometry()

Add meshes or 3D point clouds to TensorBoard. The visualization is based on Three.js, so it allows users to interact with the rendered object. Besides the basic definitions such as vertices, faces, users can further provide camera parameter, scene background.

Arguments:

Name	Shape	Type	Description
tag		string	Data identifier
vertices	(B, #vertices, 3)	torch.float	List of the 3D coordinates of vertices.
vert_colors	(B, #vertices, 3)	torch.uint8	List of colors from 0 to 255 for each vertex.
faces	(B, #faces, 3)	torch.int	Indices of vertices within each triangle. (Optional)
face_color	(B, 3)	torch.uint8	List of colors for each sample in range [0,255]. (Optional)
features	(B, #vertices, 3)	torch.float	feature vectors for each vertex (Optional)
feat_colors	(B, #vertices, 3)	torch.uint8	List of colors from 0 to 255 for each feature. (Optional)
config_dict		dict	Dictionary with ThreeJS configuration. (Optional)
global_step		int	Global step value to record (Optional)
walltime		float	Optional override default walltime (time.time()) seconds after epoch of event (Optional)
description		string	A longform readable description of the summary data. Markdown is supported. (Optional)

ThreeJS Config

The following configs are supported. For all colormaps look here. For more information on the camera attributes look here.

All colors must be given in the range [0, 255].

{
  vertices_cmap: 'jet',
  features_cmap: 'jet',
  mesh_color: [245, 124, 0],
  camera: {
    type: 'perspective', # either 'perspective' or 'orthografic'
    position: [x, y, z], # default is calculated to fit to objects in scene
    far: 1000, # far plane of camera
    near: 0.1, # near plane of camera
    
    # if type is perspective
    fov: 50,
    
    # if type is orthografic
    left: float, # default is calculated to fit to objects in scene
    right: float,
    top: float,
    bottom: float,
  },
  scene: {
    background_color: [240, 240, 240];
  };
}