You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Thanks for the note and good point, I didn't know about this.
One challenge I see with configuring it in the config file is that it's used to model creation. But one can later optionally run with --quantize bnb.nf4 or not. So, ideally, that swap should only take place upon calling the inference/training functions and leave the original model as is.
According to bnb documentation here:
https://huggingface.co/docs/bitsandbytes/main/optimizers
https://huggingface.co/docs/bitsandbytes/main/explanations/optimizers#stable-embedding-layer
This line could alter between bnb.nn.StableEmbedding and torch.nn.Embedding, or allow it to be configurable in config file:
litgpt/litgpt/model.py
Line 28 in a8aa4ba
There are also other places in code where torch.nn.Embedding is used.
The text was updated successfully, but these errors were encountered: