You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Not sure if I read rust code correctly but it seems Args in mistralrs-server/src/main.rs has no option for uqff. Only has model, which loads in this case meta-llama/Llama-3.2-11B-Vision-Instruct, so full tensor of base model.
Describe the bug
I ran the following, and it downloaded both uqff as well as full weights from Meta. I tried to skip -m, but it seems -m is required.
./mistralrs-server -i vision-plain -m meta-llama/Llama-3.2-11B-Vision-Instruct -a vllama --from-uqff EricB/Llama-3.2-11B-Vision-Instruct-UQFF/llama-3.2-11b-vision-hqq8.uqff
Latest commit or version
3e79d85
The text was updated successfully, but these errors were encountered: