auto-instrumentation registration (open-telemetry#1304)

* [WIP] Add instrumentation configuration

* add autoloading for auto-instrumentations using SPI

* allow autoloading and non-autoloading to work

* fix attribute

* experimental config file

* fixing invalid dependencies
- deptrac was rightly complaining that API (indirectly) depended on SDK through config/sdk. For now,
remove usage of Config\SDK\Configuration\Context
- update deptrac config to allow some new dependencies

* dont register hook manager globally or in sdk

* remove unused function, psalm ignore missing extension function

* possibly fixing type-hint
psalm doesn't complain now, so that should be good

* load config files relative to cwd

* fixing hook manager enable/disable + psalm complaints

* fixing 8.1 psalm error

* use context to pass providers to instrumentations
- make "register global" a function of Sdk, but keep the sdk builder's interface intact
- invent an API instrumentation context, similar to the one in config/sdk, to pass providers
  to instrumentations
- add an initial test of autoloading from a config file

* adding tests for sdk::registerGlobal
in passing, remove some dead code for handling invalid booleans - config already handles this correctly

* linting

* test coverage for globals

* add opentelemetry extension to developer image and actions

* always register instrumentations via SPI

* register globals initializer for file-config sdk
allow SDK created from config file to coexist with components using globals initializer

* linting

* remove globals init function

* fix phan warning

* simplify hook manager
- drop storage from hook manager: can't guarantee that something else, eg swoole, won't modify storage
- drop context from hook manager: we must lazy-load globals via initializers, because not all instrumentations use SPI (although that may change in future)
- default hook manager to enabled

* add todo to deprecate Registry in future

* autoload instrumentations without config
if no config provided, still try to load them. wrap registration in a try/catch/log

* fixing phan ref, update doc

* remove phan suppress

* fix example

* bump SPI to 0.2.1

* adding late-binding tracer+provider
per review from Nevay, this will allow instrumentations to get things from Globals as late as possible

* adding late binding logger and meter providers

* more late binding test coverage

* tidy

* dont use CoversMethod yet
not available in phpunit 10, so comment out and leave a todo

* kitchen sink, remove unused var

* instrumentation config as a map, per config file spec

* adding general config

* move general config into sdk

* test config caching

* move general instrumentation config to api

* avoid bad version of sebastian/exporter

* bump config yaml files to 0.3

* fix tests

* disable hook manager during file-based init

* cleanup

* support multiple config files in autoloader

* move hook manager enable/disable out of extension hook manager
The most obvious place to do this is in a class named HookManager, which _was_ an
interface for hook managers. Rename existing HookManager to HookManagerInterface,
then re-purpose HookManager for globally enabling/disabling hook managers as well
as determining the disabled status.

* update spi config

---------

Co-authored-by: Tobias Bachert <[email protected]>

.github/workflows/php.yml

@@ -21,7 +21,7 @@ jobs:
             experimental: true
             composer_args: "--ignore-platform-reqs"
     env:
-      extensions: ast, grpc, protobuf
+      extensions: ast, grpc, opentelemetry, protobuf
 
     steps:
     - name: Set cache key

.phan/config.php

@@ -373,6 +373,7 @@
         'vendor/composer',
         'vendor/grpc/grpc/src/lib',
         'vendor/guzzlehttp',
+        'vendor/tbachert/spi/src',
         'vendor/psr',
         'vendor/php-http',
         'vendor/phpunit/phpunit/src',

composer.json

@@ -19,7 +19,7 @@
         "symfony/config": "^5.4 || ^6.4 || ^7.0",
         "symfony/polyfill-mbstring": "^1.23",
         "symfony/polyfill-php82": "^1.26",
-        "tbachert/spi": "^0.2"
+        "tbachert/spi": ">= 0.2.1"
     },
     "config": {
         "sort-packages": true,
@@ -101,6 +101,7 @@
         "phpstan/phpstan-mockery": "^1.1",
         "phpstan/phpstan-phpunit": "^1.3",
         "phpunit/phpunit": "^10 || ^11",
+        "sebastian/exporter": "<= 6.0.1 || >= 6.1.3",
         "symfony/http-client": "^5.2",
         "symfony/var-exporter": "^5.4 || ^6.4 || ^7.0",
         "symfony/yaml": "^5.4 || ^6.4 || ^7.0"
@@ -148,7 +149,18 @@
                 "OpenTelemetry\\Config\\SDK\\ComponentProvider\\Logs\\LogRecordExporterConsole",
                 "OpenTelemetry\\Config\\SDK\\ComponentProvider\\Logs\\LogRecordExporterOtlp",
                 "OpenTelemetry\\Config\\SDK\\ComponentProvider\\Logs\\LogRecordProcessorBatch",
-                "OpenTelemetry\\Config\\SDK\\ComponentProvider\\Logs\\LogRecordProcessorSimple"
+                "OpenTelemetry\\Config\\SDK\\ComponentProvider\\Logs\\LogRecordProcessorSimple",
+
+                "OpenTelemetry\\Config\\SDK\\ComponentProvider\\Instrumentation\\General\\HttpConfigProvider",
+                "OpenTelemetry\\Config\\SDK\\ComponentProvider\\Instrumentation\\General\\PeerConfigProvider",
+
+                "OpenTelemetry\\Example\\ExampleConfigProvider"
+            ],
+            "OpenTelemetry\\API\\Instrumentation\\AutoInstrumentation\\HookManagerInterface": [
+              "OpenTelemetry\\API\\Instrumentation\\AutoInstrumentation\\ExtensionHookManager"
+            ],
+            "OpenTelemetry\\API\\Instrumentation\\AutoInstrumentation\\Instrumentation": [
+                "OpenTelemetry\\Example\\ExampleInstrumentation"
             ]
         }
     }

deptrac.yaml

@@ -25,6 +25,10 @@ deptrac:
           collectors:
               -   type: directory
                   value: src/SDK/.*
+      -   name: ConfigSDK
+          collectors:
+              -   type: directory
+                  value: src/Config/SDK/.*
       -   name: Context
           collectors:
               -   type: directory
@@ -85,6 +89,18 @@ deptrac:
                 value: ^GuzzleHttp\\*
               - type: className
                 value: ^Buzz\\*
+      -   name: SPI
+          collectors:
+              - type: className
+                value: ^Nevay\\SPI\\*
+      -   name: SymfonyConfig
+          collectors:
+              - type: className
+                value: ^Symfony\\Component\\Config\\*
+              - type: className
+                value: ^Symfony\\Component\\Yaml\\*
+              - type: className
+                value: ^Symfony\\Component\\VarExporter\\*
       -   name: RamseyUuid
           collectors:
               - type: className
@@ -94,16 +110,29 @@ deptrac:
       Context:
           - FFI
       SemConv: ~
+      ConfigSDK:
+          - SymfonyConfig
+          - API
+          - SDK
+          - SPI
+          - PsrLog
+          - Composer
+          - Context
+          - Contrib
+          - Extension
       API:
           - Context
           - PsrLog
+          - SPI
       SDK:
           - +API
+          - ConfigSDK
           - SemConv
           - PsrHttp
           - HttpPlug
           - Composer
           - HttpClients
+          - SPI
           - RamseyUuid
       Contrib:
           - +SDK

docker/Dockerfile

@@ -11,6 +11,7 @@ RUN chmod +x /usr/local/bin/install-php-extensions \
         grpc \
         intl\
         opcache \
+        opentelemetry \
         pcntl \
         protobuf \
         sockets \

examples/instrumentation/configure_instrumentation.php

@@ -0,0 +1,34 @@
+<?php
+
+declare(strict_types=1);
+
+namespace _;
+
+use Nevay\SPI\ServiceLoader;
+use OpenTelemetry\API\Globals;
+use OpenTelemetry\API\Instrumentation\AutoInstrumentation\ExtensionHookManager;
+use OpenTelemetry\API\Instrumentation\AutoInstrumentation\Instrumentation;
+use OpenTelemetry\API\Logs\NoopLoggerProvider;
+use OpenTelemetry\API\Metrics\Noop\NoopMeterProvider;
+use OpenTelemetry\Config\SDK\Configuration;
+use OpenTelemetry\Config\SDK\Configuration\Context;
+use OpenTelemetry\Example\Example;
+use const PHP_EOL;
+
+/**
+ * This example uses SPI (see root composer.json extra.spi) to configure an example auto-instrumentation from a YAML file
+ */
+// EXAMPLE_INSTRUMENTATION_SPAN_NAME=test1234 php examples/instrumentation/configure_instrumentation.php
+
+require __DIR__ . '/../../vendor/autoload.php';
+
+Configuration::parseFile(__DIR__ . '/otel-sdk.yaml')->create(new Context())->setAutoShutdown(true)->buildAndRegisterGlobal();
+$configuration = \OpenTelemetry\Config\SDK\Instrumentation::parseFile(__DIR__ . '/otel-sdk.yaml')->create();
+$hookManager = new ExtensionHookManager();
+$context = new \OpenTelemetry\API\Instrumentation\AutoInstrumentation\Context(Globals::tracerProvider(), new NoopMeterProvider(), new NoopLoggerProvider());
+
+foreach (ServiceLoader::load(Instrumentation::class) as $instrumentation) {
+    $instrumentation->register($hookManager, $configuration, $context);
+}
+
+echo (new Example())->test(), PHP_EOL;

examples/instrumentation/configure_instrumentation_global.php

@@ -0,0 +1,20 @@
+<?php
+
+declare(strict_types=1);
+
+namespace _;
+
+use OpenTelemetry\Example\Example;
+use const PHP_EOL;
+
+/**
+ * This example uses SPI (see root composer.json extra.spi) to configure an example auto-instrumentation from a YAML file.
+ * The YAML file paths are relative to the current working directory.
+ */
+// EXAMPLE_INSTRUMENTATION_SPAN_NAME=test1234 php examples/instrumentation/configure_instrumentation_global.php
+putenv('OTEL_PHP_AUTOLOAD_ENABLED=true');
+putenv('OTEL_EXPERIMENTAL_CONFIG_FILE=examples/instrumentation/otel-sdk.yaml');
+
+require __DIR__ . '/../../vendor/autoload.php';
+
+echo (new Example())->test(), PHP_EOL;

examples/instrumentation/otel-sdk.yaml

@@ -0,0 +1,12 @@
+file_format: '0.3'
+
+tracer_provider:
+  processors:
+  - simple:
+      exporter:
+        console: {}
+
+instrumentation:
+  php:
+    example_instrumentation:
+      span_name: ${EXAMPLE_INSTRUMENTATION_SPAN_NAME:-example span}

examples/load_config.yaml

@@ -1,4 +1,4 @@
-file_format: '0.1'
+file_format: '0.3'
 
 resource:
   attributes:

examples/load_config_env.yaml

@@ -1,4 +1,4 @@
-file_format: '0.1'
+file_format: '0.3'
 
 disabled: ${OTEL_SDK_DISABLED}
 

examples/src/Example.php

@@ -0,0 +1,14 @@
+<?php
+
+declare(strict_types=1);
+
+namespace OpenTelemetry\Example;
+
+final class Example
+{
+
+    public function test(): int
+    {
+        return 42;
+    }
+}

examples/src/ExampleConfig.php

@@ -0,0 +1,17 @@
+<?php
+
+declare(strict_types=1);
+
+namespace OpenTelemetry\Example;
+
+use OpenTelemetry\API\Instrumentation\AutoInstrumentation\InstrumentationConfiguration;
+
+final class ExampleConfig implements InstrumentationConfiguration
+{
+
+    public function __construct(
+        public readonly string $spanName,
+        public readonly bool $enabled = true,
+    ) {
+    }
+}

examples/src/ExampleConfigProvider.php

@@ -0,0 +1,50 @@
+<?php
+
+declare(strict_types=1);
+
+namespace OpenTelemetry\Example;
+
+use OpenTelemetry\API\Instrumentation\AutoInstrumentation\InstrumentationConfiguration;
+use OpenTelemetry\Config\SDK\Configuration\ComponentProvider;
+use OpenTelemetry\Config\SDK\Configuration\ComponentProviderRegistry;
+use OpenTelemetry\Config\SDK\Configuration\Context;
+use OpenTelemetry\Config\SDK\Configuration\Validation;
+use Symfony\Component\Config\Definition\Builder\ArrayNodeDefinition;
+
+/**
+ * @implements ComponentProvider<InstrumentationConfiguration>
+ */
+final class ExampleConfigProvider implements ComponentProvider
+{
+
+    /**
+     * @psalm-suppress MoreSpecificImplementedParamType
+     * @param array{
+     *     span_name: string,
+     *     enabled: bool,
+     * } $properties
+     */
+    public function createPlugin(array $properties, Context $context): InstrumentationConfiguration
+    {
+        return new ExampleConfig(
+            spanName: $properties['span_name'],
+            enabled: $properties['enabled'],
+        );
+    }
+
+    /**
+     * @psalm-suppress UndefinedInterfaceMethod,PossiblyNullReference
+     */
+    public function getConfig(ComponentProviderRegistry $registry): ArrayNodeDefinition
+    {
+        $root = new ArrayNodeDefinition('example_instrumentation');
+        $root
+            ->children()
+                ->scalarNode('span_name')->isRequired()->validate()->always(Validation::ensureString())->end()->end()
+            ->end()
+            ->canBeDisabled()
+        ;
+
+        return $root;
+    }
+}

examples/src/ExampleInstrumentation.php

@@ -0,0 +1,52 @@
+<?php
+
+declare(strict_types=1);
+
+namespace OpenTelemetry\Example;
+
+use Exception;
+use OpenTelemetry\API\Instrumentation\AutoInstrumentation\ConfigurationRegistry;
+use OpenTelemetry\API\Instrumentation\AutoInstrumentation\Context as InstrumentationContext;
+use OpenTelemetry\API\Instrumentation\AutoInstrumentation\HookManagerInterface;
+use OpenTelemetry\API\Instrumentation\AutoInstrumentation\Instrumentation;
+use OpenTelemetry\API\Trace\Span;
+use OpenTelemetry\Context\Context;
+
+final class ExampleInstrumentation implements Instrumentation
+{
+
+    public function register(HookManagerInterface $hookManager, ConfigurationRegistry $configuration, InstrumentationContext $context): void
+    {
+        $config = $configuration->get(ExampleConfig::class) ?? throw new Exception('example instrumentation must be configured');
+        if (!$config->enabled) {
+            return;
+        }
+
+        $tracer = $context->tracerProvider->getTracer('example-instrumentation');
+
+        $hookManager->hook(
+            Example::class,
+            'test',
+            static function () use ($tracer, $config): void {
+                $context = Context::getCurrent();
+
+                $span = $tracer
+                    ->spanBuilder($config->spanName)
+                    ->setParent($context)
+                    ->startSpan();
+
+                Context::storage()->attach($span->storeInContext($context));
+            },
+            static function (): void {
+                if (!$scope = Context::storage()->scope()) {
+                    return;
+                }
+
+                $scope->detach();
+
+                $span = Span::fromContext($scope->context());
+                $span->end();
+            }
+        );
+    }
+}

psalm.xml.dist

@@ -27,6 +27,11 @@
                 <referencedClass name="GMP"/>
             </errorLevel>
         </UndefinedClass>
+        <UndefinedFunction>
+            <errorLevel type="suppress">
+                <referencedFunction name="OpenTelemetry\Instrumentation\hook"/>
+            </errorLevel>
+        </UndefinedFunction>
         <ArgumentTypeCoercion>
             <errorLevel type="suppress">
                 <directory name="./examples"/>

src/API/Globals.php

@@ -6,7 +6,7 @@
 
 use function assert;
 use Closure;
-use const E_USER_WARNING;
+use OpenTelemetry\API\Behavior\LogsMessagesTrait;
 use OpenTelemetry\API\Instrumentation\Configurator;
 use OpenTelemetry\API\Instrumentation\ContextKeys;
 use OpenTelemetry\API\Logs\EventLoggerProviderInterface;
@@ -17,13 +17,14 @@
 use OpenTelemetry\Context\Propagation\TextMapPropagatorInterface;
 use function sprintf;
 use Throwable;
-use function trigger_error;
 
 /**
  * Provides access to the globally configured instrumentation instances.
  */
 final class Globals
 {
+    use LogsMessagesTrait;
+
     /** @var Closure[] */
     private static array $initializers = [];
     private static ?self $globals = null;
@@ -67,6 +68,7 @@ public static function eventLoggerProvider(): EventLoggerProviderInterface
      *
      * @internal
      * @psalm-internal OpenTelemetry
+     * @todo In a future (breaking) change, we can remove `Registry` and globals initializers, in favor of SPI.
      */
     public static function registerInitializer(Closure $initializer): void
     {
@@ -90,7 +92,7 @@ private static function globals(): self
                 try {
                     $configurator = $initializer($configurator);
                 } catch (Throwable $e) {
-                    trigger_error(sprintf("Error during opentelemetry initialization: %s\n%s", $e->getMessage(), $e->getTraceAsString()), E_USER_WARNING);
+                    self::logWarning(sprintf("Error during opentelemetry initialization: %s\n%s", $e->getMessage(), $e->getTraceAsString()));
                 }
             }
         } finally {