Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Quantized Model #17

Open
JoyBoy-Su opened this issue Jul 11, 2024 · 5 comments
Open

Quantized Model #17

JoyBoy-Su opened this issue Jul 11, 2024 · 5 comments
Labels
inference Something about inference model About model updating

Comments

@JoyBoy-Su
Copy link
Collaborator

Speeding up Inference with Quantized models
@JoyBoy-Su JoyBoy-Su added model About model updating inference Something about inference labels Jul 11, 2024
@dhruv1710
Copy link

dhruv1710 commented Jul 11, 2024 •
edited
Loading

Can I work on it?
and we need to use dynamic quantization of pytorch right?

@EthanC111
Copy link
Collaborator

Of course! Thanks a lot! You could submit a PR!

@dhruv1710
Copy link

PR created #21

@CoffeeShifter
Copy link

What's the status of this? want to use quantized model on Windows

@dhruv1710
Copy link

I am currently adding a feature so that you can use -q and you'll be able to use a quantised model for inference

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
inference Something about inference model About model updating
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@dhruv1710 @JoyBoy-Su @EthanC111 @CoffeeShifter