-
Notifications
You must be signed in to change notification settings - Fork 412
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
fix(llmobs): avoid raising errors during llmobs integration span proc…
…essing (#10713) This PR does 2 things: ### User facing changes - captures any integration-specific `_llmobs_set_tags()` method errors and logs the error instead of potentially crashing the user application. ### Non-user facing changes Refactors the `BaseLLMIntegration` class and child classes to follow a cleaner and shared `llmobs_set_tags()` method, which internally try/catches an abstract method `_llmobs_set_tags()` instead (which is implemented by each integration). We also no longer need to check `integration.is_pc_sampled_llmobs(span)` since we don't currently do any sampling yet and we can handle it in the `llmobs_set_tags()` method if needed. tldr: `_llmobs_set_tags()` is now an abstract method that needs to be implemented by all LLM integrations, and its function signature now takes in the following arguments/keyword arguments (same as `llmobs_set_tags()`): - span: span to annotate - args: list of args passed to the traced method - kwargs: dict of keyword args passed to the traced method. If any integration requires additional data not contained by either args/kwargs (such as the model instance in Gemini or tool_input dictionary in langchain), we can pass it into the method using the kwarg dict. - response: returned response from llm provider (streamed or non-streamed) - operation: string denoting which LLM operation it is (eg. "completion", "chat", "embedding", "chain", "retrieval") I did some refactoring to each integration to follow this new signature, which included merging logic for how we handle streamed responses, and additional required args (i.e. model instance, tool inputs). Previously each integration did its own thing for `llmobs_set_tags()` with arbitrary args/kwargs, and it was difficult to maintain. Now that we have a strict function signature, future integrations should be simpler to create, and existing integrations should be easier to maintain. ## Checklist - [x] PR author has checked that all the criteria below are met - The PR description includes an overview of the change - The PR description articulates the motivation for the change - The change includes tests OR the PR description describes a testing strategy - The PR description notes risks associated with the change, if any - Newly-added code is easy to change - The change follows the [library release note guidelines](https://ddtrace.readthedocs.io/en/stable/releasenotes.html) - The change includes or references documentation updates if necessary - Backport labels are set (if [applicable](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting)) ## Reviewer Checklist - [x] Reviewer has checked that all the criteria below are met - Title is accurate - All changes are related to the pull request's stated goal - Avoids breaking [API](https://ddtrace.readthedocs.io/en/stable/versioning.html#interfaces) changes - Testing strategy adequately addresses listed risks - Newly-added code is easy to change - Release note makes sense to a user of the library - If necessary, author has acknowledged and discussed the performance implications of this PR as reported in the benchmarks PR comment - Backport labels are set in a manner that is consistent with the [release branch maintenance policy](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting)
- Loading branch information
Showing
16 changed files
with
286 additions
and
333 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.