-
Notifications
You must be signed in to change notification settings - Fork 162
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
genai: add system_fingerprint attribute #1355
base: main
Are you sure you want to change the base?
Conversation
- id: gen_ai.system_fingerprint | ||
stability: experimental | ||
type: string | ||
brief: A fingerprint to track any eventual change in the Generative AI environment. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not sure about this description, for reference the instrumentation that requires this attribute used System fingerprint of the system that generated the response
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I have two questions:
- Can we confirm that this is returned from more vendors/models than OpenAI?
- How often does system_fingerprint change? If its low cardinality enough, I'd suggest adding this to the metric as well as a good way to correlate between fingerprint changes and degradations/changes in app performance.
I think it's mostly OpenAI specific, not found something similar in Mistral or Anthropic APIs.
I don't know |
Can you add this to the metric
I think its worth adding this to the base set for metrics as Conditionally Required, If available : https://github.com/open-telemetry/semantic-conventions/blob/main/model/metrics/gen-ai.yaml If it exists in Mistral and Anthropic, then it makes sense to keep in in the generic namespace vs a vendor namespace. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks a lot for following up and adding this @xrmx!
Given that it seems to be OpenAI specific, it'd need to be gen_ai.openai.response.system_fingerprint
(or similar) attribute.
But before we can add it, we'd need to add the whole openai specific conventions doc - I created #1370 to track it and it's up for grabs for anyone interested.
Changes
Add
system_fingerprint
attribute togenai
to track the value from the OpenAI api response as requested in https://github.com/open-telemetry/opentelemetry-python-contrib/pull/2759/files/6383978266d85eb058fd8030e8da2b83e6d90043#r1710313834Merge requirement checklist
[chore]