Render interface changes: Batch processing (single render call)

[mikke89]

I am currently working on the new filter and shader effects for RmlUi 6.0, and want to ask for some API feedback in that regard. This involves expanding the render interface. Mainly, I'm considering batching the geometry and submitting all of it in a single render call at the end of Context::Render. Here are some pros and cons the way I see it:

Batch processing (single render call)

Pros:

• Much easier for the client to submit geometry in large batches to their graphics API.
• Client can quickly scan over the render commands and see where state changes need to happen, batch textures etc.
• Less hidden state between render commands (e.g. scissor, transforms).
• From RmlUi side: Ability to re-use geometry for different render commands.

Cons:

• Big breaking change, however, we could mitigate this by submitting calls to the old API as a default implementation. Opting in to the new API would simply be to override the single render method.
• Need to copy vertices and indices every frame, some more memory usage (about 30KB in demo sample).
• In principle, users can already do batch processing using the old style API, but essentially need to replicate the same batch processing (unless their graphics API already does it for them).

Here's is what the new render interface looks like using the old-style API.

And using the batched-style API, with the render commands here. Consider this a rough draft.

Any thoughts?

[Dakror]

What does Rml gain from batching the commands? Isn't there some non-negligible overhead in always gathering up all the commands in a vector to only iterate on the other side again?
Why do we gain the ability to re-use geometry for different render commands by batching all commands?
Ideally all commands would reference their associated resource / geometry with an ID so that the client can load them into their buffers as they please when the geometry is "registered", store the addresses, and then whenever the geometry is requested to be used in some context, can lookup the information through the ID.

Another slight point to make is the ability to debug the drawing process more easily if every command is emitted at the origin within Rml and not batched at the end.

[mikke89]

You bring up a lot of good considerations. The ability to more easily debug is valuable, and something I haven't really thought about, and I'm also worried about changing the API too much. So, right now, I'm probably leaning towards not going through with this.

This was in part motivated by noticing how incredibly much faster streaming the geometry every frame is, compared to submitting it in separate VAOs (in OpenGL terms), like we do in our GL3 renderer. By batching everything up, the renderer became much, much faster. And to implement this, it was very helpful to do that on the library side.

However, this slowness is just as much an implementation issue, we can in principle batch it on the renderer end. Or we can use other techniques to not change VAO every draw call, although I think most of these require GL4. Maybe we want to move to that anyway.

The second motivation was the new API that comes with the upcoming filter and shader effects. We need the ability to render geometry with different textures, different shaders, to different targets. Sometimes we can in principle re-use the same vertices and indices, for example an element's background geometry also for its clipping mask.

We do have the compiled geometry concept though, which is basically what you describe, and which we can re-use. One sticking point is that the texture is compiled into the geometry, but we really want the ability to use it with different textures or for different render operations. What do you think about moving the texture out of the compiled geometry? Also, none of the advanced operations would work with the basic uncompiled geometry rendering, unless we duplicate pretty much the whole API, which would be horrendous.

We could also do something in-between. Compile the geometry like before, but submit all draw calls batched. This would eliminate any overhead, no need to copy the geometry every frame, just submit the compiled geometry handle.

[Dakror]

The way I see it is that updating the Geometry buffers is the most costly since it stalls all operations touching the geometries and its also the biggest chunk of data to write. So reducing the amount of geometry writes is key.

Rendering to different targets, or with different shaders is all fine and can be setup independently of updating the geometry as you say, with the exception of changing textures. The question here is do only the bound textures change or do the vertex UV values change? If the values are changing in a linear fashion, a simple offset and scale factor could be sent over per draw call to tell the UV's to perform this linear operation prior to sampling.

[mikke89]

That's interesting, I guess we have the following three approaches to reducing the number of geometry writes: Batching geometry on the library side, re-using geometry on the library side, and finally, batching it on the renderer side. So previously we would always leave it to the renderer, during the calls to CompileGeometry.

We don't normally need different UVs. Take for example background color, stencil, and e.g. backdrop-filters, gradients, or images. Only the latter ones care about UVs for example, so they could share the same geometry with color and stencil. Stencil in particular doesn't care about anything but position, so that could always be shared with others, and so on. I think it's quite rare for us to have different textures with the same geometry, so I don't think adding a set of linear operations would be a good fit in these cases I am thinking of. In those cases we can afford to duplicate the geometry.