-
Notifications
You must be signed in to change notification settings - Fork 1.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Minor features #2249
base: main
Are you sure you want to change the base?
Minor features #2249
Conversation
Hey @artemorloff. This generally looks good to me! but I do want to get it reviewed by @haileyschoelkopf, just to check if we are ok with backward compatibility. |
@haileyschoelkopf still actual? |
timeout is great new feature!! Can we please merge this? |
@haileyschoelkopf @lintangsutawika hi! seems you have already applied one of the features that I suggest. What do you think about the rest? Something I can add? |
Features:
fewshot_as_multiturn
whenapply_chat_template
is not passed ornum_fewshots=0
. Why failing the run? For zero-shot setup multiturn==simple chat template, so no error at all. If chat_template is not enabled, then throw warning and disable multiturn (as long as it is not available without chat_template)predict_only
this may mean that the filter is not to be used. No user may customize filters to usepredict_only
info to manage filters behaviourfilter_device
param from cli. There was a TODO about it. If I use another LLM as a filter, I may need to pass device that DIFFERS from one used to run the "main" LLM. Like llm-as-a-judge or LLMs to score the generationsensure_ascii
forapply_chat_template
method of TemplateAPI class. Now cyrillic symbols are stored in a valid form