Decide / document advice for native instrumentations

[jack-berg]

Discussed in the 7/11/24 Java SIG and later in this slack thread. Opening an issue to track and bring more visibility to the conversation.

We want libraries and frameworks to adopt the OpenTelemetry API for native instrumentation. The idea is that distributing instrumentation is more scalable than a centralized approach. Additionally, native instrumentations cast a wider net of possible users, since although native instrumentation should be compatible with a java agent, it shouldn't require a java agent.

We should document the patterns which are idiomatic for library authors to follow. This will reduce the cognitive load of adopting OpenTelemetry APIs for instrumentation (there's a lot of concepts and more we can do to simplify the better) and promote consistency.

There are a few key questions we need to discuss before documenting:

Should instrumentations default to GlobalOpenTelemetry.get() or OpenTelemetry.noop()?

Supposing we recommend defaulting to GlobalOpenTelemetry.get()...

• Instrumentations automatically work when the otel java agent is present, which installs an instance to GlobalOpenTelemetry
• Instrumentations may double instrument when the otel java agent is present, since the otel java agent may automatically apply bytecode instrumentation for some period of time before maintainers become aware of the duplicate instrumentation. Users with older versions of the otel java agent who upgrade their library / frameworks will be especially susceptible to this.
• We need to revise our warning against GlobalOpenTelemetry which current reads: "Due to the inherent complications around initialization order involving this class and its single global instance, we strongly recommend not using GlobalOpenTelemetry unless you have a use-case that absolutely requires it. Please favor using instances of OpenTelemetry wherever possible."
• Users are confronted with application initialization ordering issues: Calling GlobalOpenTelemetry.get() prevents future future calls to GlobalOpenTelemetry.set(), so an instrumentation is loaded before user code installs OpenTelemetry will cause an exception to be thrown. Users can opt into autoconfigure when GlobalOpenTelemetry.get() is called, but this essentially prevents full programmatic config of the SDK and installation of alternative implementations of the OpenTelemetry interface.

Supposing we recommend defaulting to OpenTelemetry.noop()...

• Instrumentations require users (or the otel java agent) to explicitly install an opentelemetry instance via something like <class>.install(OpenTelemetry), and do nothing until a user installs it
• Avoids double instrumentation when otel java agent is present
• Requires otel java instrumentation maintainers to be aware of each native instrumentation and maintain bytecode manipulation to install each. In exchange for this maintenance overhead, maintainers have precise control over whether the otel version or native version of instrumentation is used.

Should instrumentations prefer the higher order instrumentation API or the lower level trace, metrics, log bridge API?

Args in favor of the instrumentation API...

• The instrumentation API makes it easier to embody the semantic conventions with interfaces to implement which impose type safety and make it clearer which pieces of data are required
• The instrumentation API is cleaner to use, since it abstracts away a variety of low level details
• The instrumentation API makes it easy to collect data in a consistent way across signals (i.e. embody metric and trace semantic conventions)

Args in favor of the trace, metrics, log bridge API...

• The instrumentation API is not part of the specification, and would be somewhat surprising to someone familiar with OpenTelemetry APIs
• If an instrumentation is embodying experimental conventions, they would need to take a dependency on experimental instrumentation API artifacts, which can cause transitive version conflicts in a user's app

[jack-berg]

cc @open-telemetry/java-instrumentation-approvers

[zeitlinger]

explicitly install an opentelemetry instance via something like .install(OpenTelemetry), and do nothing until a user installs it

The agent could install the otel instance automatically if the instrumentation somehow signals it, e.g. by implementing an interface