-
Notifications
You must be signed in to change notification settings - Fork 1.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Refactor] Openai completions #1008
Conversation
…ation-harness into openai_completions
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Approving, although noting that @lintangsutawika flagged that
the chat version of the model tends to over-explain so it fails on our current settings without additional filters
We may want to support variants of tasks which are more tailored for verbose CoT-using models, down the line.
@haileyschoelkopf Openai's api that returns logits will be retired on Jan 4 2024 so it probably doesn't make much sense to continue to support it. Which means the only api available will be the chat completions.