Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update TGI specs (+ define TextToSpeech independently) #915

Merged
merged 5 commits into from
Sep 24, 2024

Conversation

Wauplin
Copy link
Contributor

@Wauplin Wauplin commented Sep 20, 2024

This PR pulls latest changes from TGI specs. In particular:

  • updated ChatCompletionInputMessageContent => now supports single message (as before), array of messages and sending image as input (in case of VLMs)
  • new input: stream_options
  • tools input have been updated
  • more documented fields

Note: TGI specs have a guideline input parameter for chat completion. I removed it from the official specs as it's a very specific parameter and subject to deletion in the near future (it was there to fix a chat template issue for ShieldGemma models). More context here (internal).

Note: I also took the opportunity to define TextToSpeech specs independently from TextToAudio (otherwise the semi-automatic scripts are not working properly). Nothing has changed in the specs.

I'll open a related PR on huggingface_hub side to reflect this change.

@julien-c
Copy link
Member

general comment that this would really need to be maintained by TGI's maintainers IMO... poke @OlivierDehaene @ErikKaum @Narsil

@Wauplin
Copy link
Contributor Author

Wauplin commented Sep 20, 2024

Yes, or running it in a CI since it's just a script to run now

Copy link
Collaborator

@mishig25 mishig25 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

from my side, lgtm !
checked the generated inference.ts interfaces

@Wauplin
Copy link
Contributor Author

Wauplin commented Sep 24, 2024

Thanks for checking the JS part @mishig25! I've used and checked the Python part myself so let's say it's fine to merge.

@Wauplin Wauplin merged commit 1aae6ac into main Sep 24, 2024
5 checks passed
@Wauplin Wauplin deleted the update-tgi-specs branch September 24, 2024 12:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants