Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add scripts to support onnx export #622

Merged
merged 10 commits into from
Jul 29, 2024
Merged

Add scripts to support onnx export #622

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
b98eb52
support onnx export
ZillaRU Jul 23, 2024
2b4bed8
chore(format): run black on main
github-actions[bot] Jul 23, 2024
67e33f4
Merge pull request #1 from ZillaRU/formatter-main
ZillaRU Jul 24, 2024
f665f2c
support trace models to onnx or jit model
ZillaRU Jul 28, 2024
c8ca35c
add tools/model_converter
ZillaRU Jul 28, 2024
6cbe15f
Merge branch 'dev' into main
ZillaRU Jul 28, 2024
1de6942
chdir onnx converter (tools -> examples)
ZillaRU Jul 28, 2024
51e0204
chdir onnx converter (tools -> examples)
ZillaRU Jul 28, 2024
fd55a09
sync with original repo
ZillaRU Jul 28, 2024
8bd6b17
Merge branch 'dev' into main
fumiama Jul 29, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 17 additions & 0 deletions onnx_export/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
# Export onnx models for deployment

## Export GPT
1. Downgrade the transformers package to v4.32.
- `pip install transformers==4.32`
- remove `from transformers.cache_utils import Cache` in `./ChatTTS/model/gpt.py`

2. Replace `modeling_llama.py` with `./onnx_export/assets/modeling_llama.py`.
ZillaRU marked this conversation as resolved.
Show resolved Hide resolved
Run `pip show transformers` to get the path to the transformers package, and run `cp ./onnx_export/assets/modeling_llama.py [path_to_transformers]/src/transformers/models/llama/modeling_llama.py`.

3. Run `trace_gpt.py`

## Export other models
Run `trace_others.py`

## Reference
[Run LLMs on Sophon TPU](https://github.com/sophgo/LLM-TPU)
Loading
Loading