Startup optimizations ideas

[OndroMih]

Hi,

I'm thinking how Piranha stands in the serverless and scalable microservices worlds where fast startup times are required. Especially in serverless, where each request processing can lead to a new startup.

I'd like to explore options, in these categories:

• runtime optimization - e.g. algorithm optimizations, caching (e.g. reading metadata like annotations)
• build-time optimizations - e.g. bytecode manipulation or annotation scanning into an index (I saw that this is already implemented to some extent with piranha.idx index but not documented, and I think it happens during runtime, so falls more or less into the first category)
• native compilation (using GraalVM)

I think that, although runtime optimizations are the most natural ones and the only ones that made sense in traditional application servers, build-time optimizations make more sense in cloud scenarios as they shift some work to happen before application is started. And native compilation is an extreme form of build-time optimization, however, not always necessary or even convenient, with its own set of challenges.

Therefore I'd like to focus on build-time optimizations mostly. They can be extended to native compilation with some additional work, for example, if reflection is replaced with bytecode manipulation or bytecode generation. Build-time optimizations could shift annotation scanning and reflection to from runtime to some time before so that optimized application is deployed/stated at runtime. I'm not sure but maybe build-time optimization could be implemented by an extension, which scans the application before it's started and then outputs an optimized application instead of stating it. The same engine could optimize an application deployed on Piranha Server so that it starts faster on next boot.

Does anybody have any thoughts on this? What options do we have to implement this? Would such optimization work as an extension?

I'd be glad for any thoughts or pointers into existing solutions so that I can investigate them and see whether I can improve them towards build-time optimization.

[Thihup]

I was thinking about this some time ago.
There are a few places where we might end up doing something:

• Create a Maven extension that unifies all web-fragments to a single web.xml (or output a Piranha Embedded that is the result of combining all of servlets/filters/etc - so the XML parsing could go away)
• Implement some custom JLink plugin to do a better "closed world" analysis when creating a custom runtime - create an index file and read it afterward or something
• See where we can go using Class Data Sharing (Hotspot) and/or change our custom classloader to be a URLClassLoader and rely on SharedClasses (OpenJ9)

Project Leyden seems to targgeting JLink and CDS. So maybe we could start working on it - or follow the progress of this project just for now.

But probably there are more options, these are the ones I could think of right now.

[mnriem]

@Thihup I like bullet 1 as that is a very attainable goal.

[OndroMih]

I agree, bullet 1 makes a lot of sense. I would build it as a separate component, not tied to any technology, so that it can be used in a maven plugin, gradle plugin, command line or even as a Piranha extension (Piranha could just output an optimized app instead of running it). It could for example build a Jandex index that would be parsed by the AnnotationScanning extension.

I also like the idea of supporting CDS. Piranha Embedded doesn't need an URLClassLoader-based classloader already, as it uses the Platform classloader. It just needs to know where to find classes and resources to scan annotations and metadata, and that could be provided by building an index on compile time, so it's basically connected to the bullet no. 1. According to my tests and also by this blog, CDS can reduce execution time of short-lived cloud functions by a significant portion (from 30% to 60%). And doesn't require any code change or a custom runtime, all can be tuned by the _JAVA_OPTIONS env variable.

I'm also investigating how to make Piranha Embedded annotation scanning work with the Platform classloader, as a fallback in case no index is provided. Weld does it using some tricks (this is why it detects annotations in @Thihup's REST template project. I'll try to improve the Piranha AnnotationScanning extension so that it uses a similar mechanism in case it doesn't run within a ResourceManagerClassLoader. I know this could be done using Jandex to create an index at build time, I'm just exploring options so that we can later discuss which is better. If we have our own scanning mechanism, we could create an index directly in bytecode, which would be faster to start than parsing the Jandex index.

Regarding using Jandex, I have one question. Why is Jandex parsed in the MicroInnerDeployer and not in the AnnotationScanning extension? It looks like MicroInnerDeployer relies on the annotationManager defined in the AnnotationScanning extension, so it must be on the classpath. But with the current solution, Jandex is only parsed in Piranha Micro and not in Piranha Embedded or Server, am I right?

[mnriem]

Jandex is indeed not used in Piranha Embedded and that is by design as Piranha Embedded itself does not do annotation scanning unless you add it manually. This is done to make Piranha Embedded completely composable.

[OndroMih]

I agree. But Jandex could be added to the AnnotationScan extension, which isn't in Piranha Embedded by default too. Currently, if AnnotationScan extension is added to Embedded, it does nothing, because it doesn't run in the ResourceManagerClassLoader. So, currently, there's no way to read annotations in Embedded. We can either improve the AnnotationScan extension to read them from Jandex, or create a new extension to do that. The only problem with a separate extension is that it would either have to include it's own AnnotationManager or depend on the AnnotationScan extension and use its InternalAnnotationScanAnnotationManager. We is your preferred way, @mnriem? 1) Enhance the AnnotationScan extension 2) separate extension with its own AnnotationManager 3) separate extension using the annotation manager from AnnotationScan extension

[mnriem]

@OndroMih Do you think we can deliver a DefaultAnnotationManager implementation that can work for both the current AnnotationScanExtension and a Jandex based one and then deliver each as a separate extension? I think that would be the best approach as it allows you to swap to either extension if so desired (e.g. if the Jandex license is a concern).

[OndroMih]

I think it's possible. But I don't know where the code for this default manager would reside. Currently, the InternalAnnotationScanAnnotationManager resides in the AnnotationScan extension, which creates it and supplies it to the web app if no manager exists. If the same logic is applied in the Jandex extension, then we'd have to duplicate the manager in both extensions. If we want to avoid that, we would have to build the default manager into the web app itself so that some manager is always present. Or add it into a separate extension, which would become a dependency of both AnnotationScan and Jandex extensions.

So, I think we have 2 options:

1. default manager provided by a separate extension

   • Webapp unchanged (no default manager)
   • AnnotationScan extension expects manager is present and doesn't provide its own InternalAnnotationScanAnnotationManager
   • a new Jandex extension expects manager and doesn't provide its own
   • a new DefaultAnnotationManager extension provides a manager (InternalAnnotationScanAnnotationManager is moved from AnnotationScan to this extension)
   • both AnnotationScan and Jandex require DefaultAnnotationManager extension already loaded, otherwise they won't do anything

2. default manager provided by Webapp

   • Webapp will contain a default manager, which is initialized for any application, or lazily the first time getAnnotationManager() is called
   • AnnotationScan extension expects manager is present and doesn't provide its own InternalAnnotationScanAnnotationManager
   • a new Jandex extension expects manager and doesn't provide its own

I think that option 2 is OK if the default manager is created lazily. It's just 2 classes and the manager wouldn't be initialized if it's not needed. Besides, it's a good pattern to always provide some implementation instead of a null result, even if the implementation is a NOOP one. If we want to keep similar performance as before with a null manager, we can introduce a new method like AnnotationManager.isEmpty or isInitialized so that the caller can find out whether the annotation manager contains some useful info or it can avoid using it altogether and save some useless method calls on the annotation manager.

Another way to implement option 2 and keep a similar performance is to keep the current behavior of the getAnnotationManager() method and introduce a new method getOrCreateAnnotationManager() that would be primarily used by extensions that want to fill in annotation info.

[mnriem]

@OndroMih Note that the WebApplicationManager API/implementation delivers the managers that a web application has access to. If we want to deliver a default annotation manager it should be non-lazily constructed there. Then each of the extensions could use the AnnotationManager API to interact with it. Feel free to come up with a refactoring with that in mind and the 2 extensions. Would love to see Jandex support :)