Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[InferenceClient] Add trust_env parameter in InferenceClient #2525

Conversation

morgandiverrez
Copy link
Contributor

@morgandiverrez morgandiverrez force-pushed the InferenceClient-new-trust_env-parameter branch from dbcc60e to 696a970 Compare September 6, 2024 17:12
@morgandiverrez
Copy link
Contributor Author

hello, @Wauplin what do you think about this PR ?

@Wauplin
Copy link
Contributor

Wauplin commented Sep 11, 2024

Hi @morgandiverrez, sorry about the long delay. I wanted to check where exactly this is an issue in langchain integration. Can you pin-point the breaking line on GitHub? I just want to check the context of it

@morgandiverrez
Copy link
Contributor Author

morgandiverrez commented Sep 11, 2024

hello, yes, in this line : https://github.com/langchain-ai/langchain/blob/398718e1cb555e526bbf76ae163ded4c8d3cd1f8/libs/partners/huggingface/langchain_huggingface/llms/huggingface_endpoint.py#L216

if we add in kwargs trust_env=true for set in asyncClient, we have an error by inferenceClient

@Wauplin
Copy link
Contributor

Wauplin commented Sep 12, 2024

Hi @morgandiverrez, thanks for the reference. First of all, sorry for the delay, I first wanted to check how to deal with this. I just opened a PR to fix things on the langchain side rather than here. See langchain-ai/langchain#26378.

While this PR is correct, I prefer not to add parameters when it's not necessary. As mentioned in langchain-ai/langchain#26378, I think that langchain integration should not assume a perfect match in supported kwargs for both clients. trust_env is a first issue but we might add new ones in the future that are only relevant to one or the other client but not both.

If the PR gets accepted, I would prefer to close this one without merging. Advanced users already have a way to configure trust_env in huggingface_hub globally (see docs) for the sync client and giving the possibility here for a new argument might be misleading (+ create side effects since we are mutating the global session, not used only in InferenceClient). If you have any remarks and questions, please let me know :)

@Wauplin
Copy link
Contributor

Wauplin commented Sep 23, 2024

As explained above, I'll close this PR without merging in favor of langchain-ai/langchain#26378 that just got merged.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants