Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Support VLM in chat completion (+some specs updates) #2556

Merged
merged 4 commits into from
Sep 25, 2024
Merged

Support VLM in chat completion (+some specs updates) #2556

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
e0d39b7
Support VLM in chat completion (+some specs updates)
Wauplin Sep 20, 2024
94ac490
document VLM
Wauplin Sep 20, 2024
e35c0ae
Merge branch 'main' into support-vlm-in-chat-completion
Wauplin Sep 25, 2024
9e824b1
idefics example
Wauplin Sep 25, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 22 additions & 6 deletions docs/source/en/package_reference/inference_types.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -63,9 +63,9 @@ This part of the lib is still under development and will be improved in future r

[[autodoc]] huggingface_hub.ChatCompletionInputMessageChunk

[[autodoc]] huggingface_hub.ChatCompletionInputTool
[[autodoc]] huggingface_hub.ChatCompletionInputStreamOptions

[[autodoc]] huggingface_hub.ChatCompletionInputToolTypeClass
[[autodoc]] huggingface_hub.ChatCompletionInputToolType

[[autodoc]] huggingface_hub.ChatCompletionInputURL

Expand Down Expand Up @@ -103,6 +103,10 @@ This part of the lib is still under development and will be improved in future r

[[autodoc]] huggingface_hub.ChatCompletionStreamOutputTopLogprob

[[autodoc]] huggingface_hub.ChatCompletionStreamOutputUsage

[[autodoc]] huggingface_hub.ToolElement



## depth_estimation
Expand Down Expand Up @@ -219,12 +223,12 @@ This part of the lib is still under development and will be improved in future r

## summarization

[[autodoc]] huggingface_hub.SummarizationGenerationParameters

[[autodoc]] huggingface_hub.SummarizationInput

[[autodoc]] huggingface_hub.SummarizationOutput

[[autodoc]] huggingface_hub.SummarizationParameters



## table_question_answering
Expand Down Expand Up @@ -307,6 +311,18 @@ This part of the lib is still under development and will be improved in future r



## text_to_speech

[[autodoc]] huggingface_hub.TextToSpeechGenerationParameters

[[autodoc]] huggingface_hub.TextToSpeechInput

[[autodoc]] huggingface_hub.TextToSpeechOutput

[[autodoc]] huggingface_hub.TextToSpeechParameters



## token_classification

[[autodoc]] huggingface_hub.TokenClassificationInput
Expand All @@ -319,12 +335,12 @@ This part of the lib is still under development and will be improved in future r

## translation

[[autodoc]] huggingface_hub.TranslationGenerationParameters

[[autodoc]] huggingface_hub.TranslationInput

[[autodoc]] huggingface_hub.TranslationOutput

[[autodoc]] huggingface_hub.TranslationParameters



## video_classification
Expand Down
32 changes: 24 additions & 8 deletions docs/source/ko/package_reference/inference_types.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -62,9 +62,9 @@ rendered properly in your Markdown viewer.

[[autodoc]] huggingface_hub.ChatCompletionInputMessageChunk

[[autodoc]] huggingface_hub.ChatCompletionInputTool
[[autodoc]] huggingface_hub.ChatCompletionInputStreamOptions

[[autodoc]] huggingface_hub.ChatCompletionInputToolTypeClass
[[autodoc]] huggingface_hub.ChatCompletionInputToolType

[[autodoc]] huggingface_hub.ChatCompletionInputURL

Expand Down Expand Up @@ -102,6 +102,10 @@ rendered properly in your Markdown viewer.

[[autodoc]] huggingface_hub.ChatCompletionStreamOutputTopLogprob

[[autodoc]] huggingface_hub.ChatCompletionStreamOutputUsage

[[autodoc]] huggingface_hub.ToolElement



## depth_estimation[[huggingface_hub.DepthEstimationInput]]
Expand Down Expand Up @@ -216,14 +220,14 @@ rendered properly in your Markdown viewer.



## summarization[[huggingface_hub.SummarizationGenerationParameters]]

[[autodoc]] huggingface_hub.SummarizationGenerationParameters
## summarization[[huggingface_hub.SummarizationInput]]

[[autodoc]] huggingface_hub.SummarizationInput

[[autodoc]] huggingface_hub.SummarizationOutput

[[autodoc]] huggingface_hub.SummarizationParameters



## table_question_answering[[huggingface_hub.TableQuestionAnsweringInput]]
Expand Down Expand Up @@ -306,6 +310,18 @@ rendered properly in your Markdown viewer.



## text_to_speech[[huggingface_hub.TextToSpeechGenerationParameters]]

[[autodoc]] huggingface_hub.TextToSpeechGenerationParameters

[[autodoc]] huggingface_hub.TextToSpeechInput

[[autodoc]] huggingface_hub.TextToSpeechOutput

[[autodoc]] huggingface_hub.TextToSpeechParameters



## token_classification[[huggingface_hub.TokenClassificationInput]]

[[autodoc]] huggingface_hub.TokenClassificationInput
Expand All @@ -316,14 +332,14 @@ rendered properly in your Markdown viewer.



## translation[[huggingface_hub.TranslationGenerationParameters]]

[[autodoc]] huggingface_hub.TranslationGenerationParameters
## translation[[huggingface_hub.TranslationInput]]

[[autodoc]] huggingface_hub.TranslationInput

[[autodoc]] huggingface_hub.TranslationOutput

[[autodoc]] huggingface_hub.TranslationParameters



## video_classification[[huggingface_hub.VideoClassificationInput]]
Expand Down
28 changes: 20 additions & 8 deletions src/huggingface_hub/__init__.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -294,8 +294,8 @@
"ChatCompletionInputGrammarType",
"ChatCompletionInputMessage",
"ChatCompletionInputMessageChunk",
"ChatCompletionInputTool",
"ChatCompletionInputToolTypeClass",
"ChatCompletionInputStreamOptions",
"ChatCompletionInputToolType",
"ChatCompletionInputURL",
"ChatCompletionOutput",
"ChatCompletionOutputComplete",
Expand All @@ -314,6 +314,7 @@
"ChatCompletionStreamOutputLogprob",
"ChatCompletionStreamOutputLogprobs",
"ChatCompletionStreamOutputTopLogprob",
"ChatCompletionStreamOutputUsage",
"DepthEstimationInput",
"DepthEstimationOutput",
"DocumentQuestionAnsweringInput",
Expand Down Expand Up @@ -348,9 +349,9 @@
"QuestionAnsweringParameters",
"SentenceSimilarityInput",
"SentenceSimilarityInputData",
"SummarizationGenerationParameters",
"SummarizationInput",
"SummarizationOutput",
"SummarizationParameters",
"TableQuestionAnsweringInput",
"TableQuestionAnsweringInputData",
"TableQuestionAnsweringOutputElement",
Expand Down Expand Up @@ -379,12 +380,17 @@
"TextToImageOutput",
"TextToImageParameters",
"TextToImageTargetSize",
"TextToSpeechGenerationParameters",
"TextToSpeechInput",
"TextToSpeechOutput",
"TextToSpeechParameters",
"TokenClassificationInput",
"TokenClassificationOutputElement",
"TokenClassificationParameters",
"TranslationGenerationParameters",
"ToolElement",
"TranslationInput",
"TranslationOutput",
"TranslationParameters",
"VideoClassificationInput",
"VideoClassificationOutputElement",
"VideoClassificationParameters",
Expand Down Expand Up @@ -802,8 +808,8 @@ def __dir__():
ChatCompletionInputGrammarType, # noqa: F401
ChatCompletionInputMessage, # noqa: F401
ChatCompletionInputMessageChunk, # noqa: F401
ChatCompletionInputTool, # noqa: F401
ChatCompletionInputToolTypeClass, # noqa: F401
ChatCompletionInputStreamOptions, # noqa: F401
ChatCompletionInputToolType, # noqa: F401
ChatCompletionInputURL, # noqa: F401
ChatCompletionOutput, # noqa: F401
ChatCompletionOutputComplete, # noqa: F401
Expand All @@ -822,6 +828,7 @@ def __dir__():
ChatCompletionStreamOutputLogprob, # noqa: F401
ChatCompletionStreamOutputLogprobs, # noqa: F401
ChatCompletionStreamOutputTopLogprob, # noqa: F401
ChatCompletionStreamOutputUsage, # noqa: F401
DepthEstimationInput, # noqa: F401
DepthEstimationOutput, # noqa: F401
DocumentQuestionAnsweringInput, # noqa: F401
Expand Down Expand Up @@ -856,9 +863,9 @@ def __dir__():
QuestionAnsweringParameters, # noqa: F401
SentenceSimilarityInput, # noqa: F401
SentenceSimilarityInputData, # noqa: F401
SummarizationGenerationParameters, # noqa: F401
SummarizationInput, # noqa: F401
SummarizationOutput, # noqa: F401
SummarizationParameters, # noqa: F401
TableQuestionAnsweringInput, # noqa: F401
TableQuestionAnsweringInputData, # noqa: F401
TableQuestionAnsweringOutputElement, # noqa: F401
Expand Down Expand Up @@ -887,12 +894,17 @@ def __dir__():
TextToImageOutput, # noqa: F401
TextToImageParameters, # noqa: F401
TextToImageTargetSize, # noqa: F401
TextToSpeechGenerationParameters, # noqa: F401
TextToSpeechInput, # noqa: F401
TextToSpeechOutput, # noqa: F401
TextToSpeechParameters, # noqa: F401
TokenClassificationInput, # noqa: F401
TokenClassificationOutputElement, # noqa: F401
TokenClassificationParameters, # noqa: F401
TranslationGenerationParameters, # noqa: F401
ToolElement, # noqa: F401
TranslationInput, # noqa: F401
TranslationOutput, # noqa: F401
TranslationParameters, # noqa: F401
VideoClassificationInput, # noqa: F401
VideoClassificationOutputElement, # noqa: F401
VideoClassificationParameters, # noqa: F401
Expand Down
Loading
Loading