Gen Prefix #2274
Open
Gen Prefix #2274
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
baberabb
requested review from
haileyschoelkopf and
lintangsutawika
as code owners
baberabb
commented
Sep 2, 2024
| labeled_examples, | ||
| example, | ||
| fewshot_as_multiturn, | ||
| gen_prefix, | ||
| ) | ||
| # for loglikelihood create a list of questions with appended choices | ||
| elif isinstance(example, list): |
There was a problem hiding this comment.
Haven't been able to identify when non-chat_template example is a list
There was a problem hiding this comment.
it would be better to split fewshot_context func into two: one to make a list of samples, another to make it a valid string
right now i am working on smart truncation that allows users to choose what to truncate and I encountered the same problem
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
closes #2248. Adds a
gen_prefixto the assistant message or appends it in case of completion.Added a couple of TODO comments where I wasn't sure how to proceed.
Also not sure how to deal with unsupported APIs as yet. The
gen_prefixis set in the task yaml. We could add a warning and remove it in the models chat template.