Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add system specific conventions for OpenAI #1385

Merged
merged 9 commits into from
Oct 2, 2024

Conversation

drewby
Copy link
Member

@drewby drewby commented Sep 4, 2024

Fixes #1370

Changes

Adds system specific attributes for OpenAI.

  • gen_ai.openai.request.seed
  • gen_ai.openai.request.response_format
  • gen_ai.openai.request.json_schema.name
  • gen_ai.openai.request.service_tier
  • gen_ai.openai.response.service_tier

Adds an openai.md document to outline the attributes that should be added to spans in addition to the general attributes. It also outlines the attributes which should be added to generative AI metrics.

Merge requirement checklist

@drewby drewby self-assigned this Sep 4, 2024
@drewby
Copy link
Member Author

drewby commented Sep 4, 2024

I'm creating this as Draft since its the first vendor specific conventions we are adding to the gen_ai area.
For spans, its a list of the new attributes for OpenAI and the general attributes combined.
For metrics, its only the OpenAI attributes and refers to the general metric document for other attributes.

  • Should these be in one file or separate files for metrics and spans?
  • Should it be a combined list of openai and general attributes, or only the openai attributes that get added?

Copy link
Contributor

@lmolkova lmolkova left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you! Left some comments

docs/gen-ai/openai.md Outdated Show resolved Hide resolved
docs/gen-ai/openai.md Outdated Show resolved Hide resolved
docs/gen-ai/openai.md Outdated Show resolved Hide resolved
docs/gen-ai/openai.md Outdated Show resolved Hide resolved
docs/gen-ai/openai.md Outdated Show resolved Hide resolved
docs/gen-ai/openai.md Show resolved Hide resolved
model/registry/gen-ai.yaml Outdated Show resolved Hide resolved
jack-berg pushed a commit to jack-berg/semantic-conventions that referenced this pull request Sep 4, 2024
jack-berg pushed a commit to jack-berg/semantic-conventions that referenced this pull request Sep 4, 2024
@lmolkova
Copy link
Contributor

lmolkova commented Sep 5, 2024

I've sent a similar one for Azure AI Inference - #1393 (without metrics though - no plans to report anything custom there yet).

Should these be in one file or separate files for metrics and spans?

Either way works for me. I'd keep them in one file until it grows too big.

Should it be a combined list of openai and general attributes, or only the openai attributes that get added?

Yes, the point is to describe all attributes that are applicable (and how if it's not obvious). Otherwise it's not clear which generic ones apply to openai.

@drewby
Copy link
Member Author

drewby commented Sep 6, 2024

Yes, the point is to describe all attributes that are applicable (and how if it's not obvious). Otherwise it's not clear which generic ones apply to openai.

Thanks, I brought this up in the Azure inference list as well, but is there a way to highlight, sort, or otherwise make it clear which ones are specific to openai? One can scan the list and fine the namespaces, but it'd be easier if they stood out visually.

model/trace/gen-ai.yaml Outdated Show resolved Hide resolved
@drewby drewby marked this pull request as ready for review September 25, 2024 17:17
@drewby drewby requested review from a team as code owners September 25, 2024 17:17
Copy link
Contributor

@karthikscale3 karthikscale3 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me. Thanks @drewby

Copy link

@codefromthecrypt codefromthecrypt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Only glitch is service tiers, and I wonder if there would be difference vs OpenAI the platform vs Azure OpenAI. Certainly emulators would have differences. Should we mention the openai api docs, but caution that platform differences apply in exact values?

docs/gen-ai/openai.md Show resolved Hide resolved
model/registry/gen-ai.yaml Outdated Show resolved Hide resolved
model/registry/gen-ai.yaml Outdated Show resolved Hide resolved
Copy link
Contributor

@lmolkova lmolkova left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left some minor comments, but LGTM overall!

model/trace/gen-ai.yaml Outdated Show resolved Hide resolved
model/trace/gen-ai.yaml Outdated Show resolved Hide resolved
docs/gen-ai/openai.md Outdated Show resolved Hide resolved
docs/gen-ai/openai.md Outdated Show resolved Hide resolved
docs/gen-ai/openai.md Outdated Show resolved Hide resolved
model/metrics/gen-ai.yaml Outdated Show resolved Hide resolved
model/registry/gen-ai.yaml Outdated Show resolved Hide resolved
model/trace/gen-ai.yaml Outdated Show resolved Hide resolved
@drewby drewby requested review from a team as code owners October 1, 2024 08:05
@drewby drewby requested review from a team and tigrannajaryan as code owners October 1, 2024 08:05
@drewby drewby closed this Oct 1, 2024
@drewby drewby reopened this Oct 1, 2024
@lmolkova lmolkova merged commit 87ec4e6 into open-telemetry:main Oct 2, 2024
14 checks passed
ChrsMark pushed a commit to ChrsMark/semantic-conventions that referenced this pull request Oct 18, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Development

Successfully merging this pull request may close these issues.

GenAI: Add system-specific conventions
7 participants