Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Volume rendering compositing changes and wireframe rendering to vis chunks in volume rendering mode #503

Merged
merged 19 commits into from
Feb 6, 2024

Conversation

seankmartin
Copy link
Contributor

Performs front-to-back compositing in three steps:

  1. The opacity from the current data point is corrected based on the ratio between the optimal number of depth samples to use for the current data resolution, and the actual number of chosen depth samples. This avoids the output image changing opacity if the volume is over/undersampled.
  2. The corrected opacity is composited with the already accumulated opacity along the ray.
  3. The resulting composited opacity is multiplied by the already accumulated color, and then the accumulated color and opacity information is updated with the opacity multiplied color and the composited opacity.

Also adds a chunk visualiser to VR if wireframe mode is enabled in settings

@seankmartin seankmartin marked this pull request as draft December 27, 2023 07:41
@seankmartin seankmartin marked this pull request as ready for review January 31, 2024 17:46
@seankmartin
Copy link
Contributor Author

This pull request implements four main features:

  1. OIT is performed along rays during volume rendering ray marching. This is to try and remove chunk border artifacts.
  2. During ray marching, the value stored in the offscreen Z buffer is compared against the depth computed at the current ray position. If the ray has been deemed to pass behind an opaque object, the compositing loop for that ray ends.
  3. Wireframe rendering mode shows volume rendering chunks by a different color.
  4. A gain parameter has been included in volume rendering mode, to heighten or lessen the opacity value by scaling with the gain.

Additionally, the volume rendering flag has been bound to a Python controllable tool.

I think it should be good for another round of review if you get the time @jbms. Thanks again for all the thoughtful input on this one so far, really helpful and much appreciated!


const OPACITY_JSON_KEY = "opacity";
const GAIN_JSON_KEY = "gain";
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe rename to volumeRenderingGain since it only applies to volume rendering?

Also when I've created this control manually as a UI control, I've typically found it convenient to give it an exponential scale so that it can more easily cover a larger range, i.e. the slider ranges from -10 to 10 and the actual gain is exp(gain). But that could also be handled purely in the slider control. Maybe you've also found the linear gain to work okay.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great, thanks! Linear worked pretty well, but I prefer the exponential scale after trying it out, thanks for the suggestion on that.
Not sure if we should hint at the scale in the UI somehow now, or if the name Gain (3D) is clear enough. Happy either way, but let me know if you have thoughts on that

@jbms
Copy link
Collaborator

jbms commented Feb 4, 2024

Thanks! This looks good --- I left just a few minor comments.

@@ -125,6 +127,7 @@ export class ImageUserLayer extends Base {
dataType = new WatchableValue<DataType | undefined>(undefined);
sliceViewRenderScaleHistogram = new RenderScaleHistogram();
sliceViewRenderScaleTarget = trackableRenderScaleTarget(1);
volumeRenderingGain = trackableFiniteFloat(1);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Default should probably be 0.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, good catch, sorry about that!

@jbms jbms merged commit a9939d6 into google:master Feb 6, 2024
11 of 20 checks passed
@seankmartin seankmartin deleted the feature/volume-rendering-compositing branch February 20, 2024 14:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants