Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Andreyan/exporters model configs #1159

Conversation

andreyanufr
Copy link

What does this PR do?

  1. Added configuration for onnx conversion of MPTForCausalLM architecture (https://huggingface.co/mosaicml/mpt-7b-instruct/blob/main/config.json).
  2. Changed default seq_len for correct conversion of MPTForCausalLM with use_cache=True flag (past key values)

@fxmarty
Copy link
Contributor

fxmarty commented Jul 6, 2023

Hi @andreyanufr , thank you for your contribution! MPT is currently not supported in transformers (but will be), will have a look once it is in a release.

Meanwhile, I would recommend to use custom ONNX configs following #1166

@MaximProshin
Copy link

@fxmarty , do you have any idea when MPT could be supported in hf? I wonder if we should wait for it or use custom configs as you suggested.

@regisss
Copy link
Contributor

regisss commented Jul 11, 2023

@MaximProshin You can follow the advancement here: huggingface/transformers#24629
But it's impossible to say when this PR will be merged.

@andreyanufr andreyanufr force-pushed the andreyan/exporters_model_configs branch from 395ea37 to 757ec66 Compare July 26, 2023 07:23
@fxmarty
Copy link
Contributor

fxmarty commented Sep 4, 2023

MPT ONNX & ORT support was added in the PR: #1161

@fxmarty fxmarty closed this Sep 4, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants