GaftferVDB : Add VolumeScatter

[danieldresser-ie]

Tidying up Don's old PR. In theory, this is pretty well ready to go, just missing tests.

The main question in my mind right now: how useful is PointCount mode really? In general, taking the shape from any active voxels, rather than looking at the actual voxel values, is going to give a very rough shape.

I tried asking Rob about it, and he didn't seem to think there would be much use for PointCount mode ... though what he thought would be extremely valuable would be being able to set min and max thresholds for the values in the VDB. ie: if he wanted to uniformly distribute points throughout a volume, he would want to select all voxels with a value greater than 0.001, not use all active voxels.

[johnhaddon]

Thanks Daniel - this is looking pretty good!

The main question in my mind right now: how useful is PointCount mode really?

I found it slightly useful when mucking about to test the node - didn't need to get density right in order to be able to see something. Seems like it might also be useful if you were trying to do something to a certain budget, like scattering lights, or making proxy geo? Although maybe doing a k-means after the scatter would be better for that? And I guess it's also capable of scattering within non-float grids, although I don't know what the use cases for that are.

I suppose it's a better match for the existing Scatter node if we only expose the density mode - that might be a reasonable argument for simplifying now and adding back the fixed mode later if we find a need for it?

Couple of other thoughts :

• Is it worth adding the equivalent of the Seeds.primitiveVariables plug, to allow the points to inherit values from other grids (temperature, color etc)? Or should that be a separate VolumeSampler node that can be used to sample grids onto any object?
• It would be nice to be able to fill the interior of a level set, rather than just the narrow band. There seems to be code for that here.

[danieldresser-ie]

I think I've now addressed all feedback. I've squashed everything in place, since we decided to remove PointCount mode for now, and after removing that, the code is a fair bit shorter, and it's probably simpler to just review one commit rather than having a bunch of removals of the old code.

I've added a few basic tests, and a Changes entry.

It would be nice to be able to fill the interior of a level set
Yes, but doing it properly would require copying and modifying openvdb::tools::NonUniformPointScatter, so we've decided to defer it.
Is it worth adding the equivalent of the Seeds.primitiveVariables?
Hmm, good question. It does seem like something we should be able to do - but we definitely need it as a separate node as well, so I'm not sure whether it makes sense to build it into this one. The only performance benefit would be if we had a special case for transferring the density var that you're generating from to the points ( which probably is pretty frequently useful ).

[johnhaddon]

a new node for scattering points...

This is currently in a section for the upcoming 1.2.9 release, but the PR is targeted to main for the upcoming 1.3 release. It could go into either, but since there's no rush for it, I think perhaps it makes sense to put it in 1.3, paired with the improved Scatter?

[danieldresser-ie]

Fixed the errors noted.

[danieldresser-ie]

Well, we've been saved from a horrible mistake by the CI running on both Windows and Linux. The code I was working from had used std::default_random_engine, and I hadn't even thought about how miserable that is until I saw the test fail on Windows.

The question of which generator we should use is perhaps a little complicated, but it should at least be consistent across platforms. For the moment, I've matched other similar usage in Gaffer, and used Imath's Rand32.

OpenVDB's examples use a Mersenne Twister ... if I had to guess the motivations of the previous dev, they probably looked at that, thought "2000 bytes is way more state than we need", and went with something simpler. The ideal bytes of state is probably somewhere in between the 4 I'm currently using, and 2000.

I haven't evaluated this in any deep regard - the generator I'm currently using is probably vulnerable to lattice artifacts, but I haven't evaluated how objectionable they would be.

It kind of feels like we may want to break this distribution at some point in the future anyway ... most generators have the option to directly generate floats, and having to conform to openvdb's weird API where you have to pass a generator that makes ints, together with a min and max for normalizing them, feels pretty unhelpful.

I dunno, maybe a Mersenne Twister would be more future proof?

[johnhaddon]

having to conform to openvdb's weird API where you have to pass a generator that makes ints, together with a min and max for normalizing them, feels pretty unhelpful.

I don't think that API is particularly weird is it? It's the C++ standard's UniformRandomBitGenerator, which means there are lots of available implementations, and it's easy for VDB to map to floats internally using std::uniform_real_distribution. It's the standard stuff.

the generator I'm currently using is probably vulnerable to lattice artifacts, but I haven't evaluated how objectionable they would be.

I have a dim memory of you fixing lattice artifacts in something else at some point - I don't remember the details, but it's making me think Rand32 might not be idea. Don't suppose you can dredge anything up from your memory banks?

I dunno, maybe a Mersenne Twister would be more future proof?

It seems like a better choice to me. Our usage of Imath::Rand32 predates all of the standard <random> stuff, so we were just using what was available at the time.

[danieldresser-ie]

Tried to do some fairly complete testing without spending too much time. Didn't end up creating automated tests, but based on interactive testing with 100 000 000 points:

std::default_random_engine : 3.5 seconds : Lattice artifacts
std::mersenne_twister_engine : 4.3 seconds : No perceptible artifacts
Imath::Rand32 : 2.3 seconds : Lattice artifacts
inline copy of PCG : 2.7 seconds : No perceptible artifacts

Note that in order to observe artifacts, I had to:

• generate 100 000 000 points from one voxel, covering a unit cube
• cull a tiny cube of this out ( just 0.05 units on each side )
• zoom in on that to full screen

So it's not clear to what extent lattice artifacts would arise in practice.

I don't really feel like the 60% slowdown and inflexible initialization of the Mersenne Twister is really worth it - if you want to just merge something now, I've prepped a version just pasting in 20 lines of lightweight PCG ... I think that's pretty reasonable as a stopgap? If you want to spend some more time, maybe it's worth adding PCG as a dependency?

[johnhaddon]

I've prepped a version just pasting in 20 lines of lightweight PCG ... I think that's pretty reasonable as a stopgap?

If you're sold on PCG, then we should add it to GafferHQ/dependencies properly. It's a requirement of Apache 2 that we distribute the license with it, and I don't think including it in the source code counts when we're uploading binaries. Besides, it's nice to give credit where it's due, and including it in GafferHQ/dependencies means it'll appear in the About window etc. Alternatively, we can use Mersenne now and revisit when we add parallelism, level set support etc.

[danieldresser-ie]

Assuming I got the gafferDependencies PR right, this should now work properly with pcg ( with simpler looking code because I don't need to declare my own wrapper ).

[johnhaddon]

LGTM. Out of interest, does the fancy templated-to-the-hilt C++ version of PCG give the same performance as the little C implementation you tested with originally?

[danieldresser-ie]

LGTM. Out of interest, does the fancy templated-to-the-hilt C++ version of PCG give the same performance as the little C implementation you tested with originally?

Seems to. I haven't tested particularly exhaustively, but in the 100 000 000 point test, it also took 2.7 seconds.

[danieldresser-ie]

Made a very minor update, just to the version of pcg at ImageEngine in config/ie/options ( so it matches the update to gafferDependencies )

[johnhaddon]

I've released a new dependencies version with PCG, and updated this PR with a commit to update to it. So in theory the Linux build should go through now. @ericmehl, could you upload an update for the Windows dependency package please?

[ericmehl]

The Windows dependencies package 📦 is up and I triggered a new build to check that all is well on that platform...

[johnhaddon]

Thanks @ericmehl, merging!