[service] add support for views

[codeboten]

This adds support for configuring metric views for internal service telemetry.

This follows #8119

Related to #3039 #8237 ##7532

[codecov]

Codecov Report

Patch coverage is 68.70% of modified lines.

Files Changed	Coverage
service/telemetry/config.go	56.14%
service/telemetry/metric_view.go	77.77%
service/telemetry.go	100.00%

📢 Thoughts on this report? Let us know!.

[jpkrohling]

Does this mean that views are only coming from the user-provided config, and not from components? I understand this is not the final state, but could you add some documentation on how to use this?

[codeboten]

That's a good point. I'll have to spend a bit of time trying to figure out how to give components the ability to create their own views.

[github-actions]

This PR was marked stale due to lack of activity. It will be closed in 14 days.

[codeboten]

One of the issues w/ this change is that a handful of components today configure metric views (using the opencensus registration mechanism available at runtime) wouldn't be able to do so in OTel. Currently there would be no way for a collector component to specify aggregation configuration. This is because there’s no way for components to return views to the service that’s configuring the otel go SDK.

Is this something we want components to be able to specify or is this something that should be left to users to configure?

There are 5 components (loadbalancingexporter, groupbyattrsprocessor, groupbytraceprocessor, tailsamplingprocessor, batchprocessor) that configure view.Distribution for their metrics. To get around this problem for batch processor metrics, there is a batchViews function called by the service telemetry initialization currently. This is somewhat problematic because now the service has to configure a view for a metric that may or may not be there. I wouldn’t want to configure all of the configuration this way as it would leak information from contrib components into the core.

Here are some of the options we have moving forward.

Remove component configured views when moving to Otel and only support user configured views

Pro

• Not having to support components configuring views means there's no additional info that needs to be passed from components to the service
• There are no more views that are automatically configured for end users, this can be somewhat surprising for users?

Con

• This would be a breaking change in the telemetry emitted by anyone using the metrics today. This would all be behind a feature gate so it would allow users to migrate slowly.
• It would also mean that anyone wanting aggregation other than the default aggregation would need additional configuration in their collector config.

Leave metrics that require custom aggregation using opencensus.

Pro

• Least amount of moving pieces
• Behaviour doesn't change for end users

Con

• This would mean we could never remove OC dependency

Add a mechanism for components factories to provide views

Pro

• Gives component authors and users most flexibility

Con

• Adds public API surface

Wait until hints API is available in otel go

Pro

• This provides components the ability to specify hints for bucket configuration

Con

• Currently scheduled for post GA release (Add experimental histogram advice API opentelemetry-go#4094)

@jpkrohling any thoughts on the best approach here or alternatives?

[jpkrohling]

Is this something we want components to be able to specify or is this something that should be left to users to configure?

I believe component owners are best positioned to configure how the telemetry should be exposed. I think advanced users might be interested in tweaking this, but I don't think the majority of the users of, say, load-balancing exporter would be willing to configure the views in the config to have the same level of metrics that are being exposed today.

Add a mechanism for components factories to provide views

I'd vote for this. It could be an optional interface, and if implemented by components, it's called when the service is configured.

[dashpole]

Other than default buckets for histograms, is there anything else that can be done with views that can't be done with the API itself? If this is going to be obsolete in ~a month when the hint API is added, it seems potentially worth waiting. Views being provided by libraries containing the instrumentation seems to go against their intended purpose.

Go already has an RC, so I would expect a stable release in a few weeks. The ExplicitBucketBoundaries config already has the necessary approvals to move to stable: open-telemetry/opentelemetry-specification#3694. It could possibly be part of the stable Go release if that happens...

Edit: aaaaaaand the spec PR is merged.

[codeboten]

@jpkrohling i've filled in a section under the observability doc on how users may enable configuration support. I've not added a section on configuring views specifically, but there is a link to the otel-configuration repo that contains that information. Here's an example of how this could be used:

      views:
      - selector:
          instrument_name: process_memory_rss
          instrument_type: observable_gauge
        stream:
          name: new_instrument_name
          description: new_description
          aggregation:
            last_value: {}
          attribute_keys:
            - key1
            - key2
      - selector:
          instrument_name: "*"
        stream:
          aggregation:
            drop: {}

It sounds like the work to include hints in the API will be unblocked soon, it might be worth waiting for this work to move this views PR forward, WDYT?

[jpkrohling]

If a long-term solution is around the corner, it does make sense to wait a bit.

About the view configuration: is it intended only for advanced use-cases, where users would tweak the telemetry generated by the collector, or is it something that, as a component developer, would have to provide to my users, so that they can use on the collector's config? As an advanced use-case, it looks fine to me.

[codeboten]

@jpkrohling yes I expect this to be a more advanced use-case since the hints API should fulfill the more common needs for components

[github-actions]

This PR was marked stale due to lack of activity. It will be closed in 14 days.

[github-actions]

This PR was marked stale due to lack of activity. It will be closed in 14 days.

[github-actions]

Closed as inactive. Feel free to reopen if this PR is still being worked on.