Edit tutorial comments on PEFT / LoRA

[vchiley]

It doesn't really address this issue but it allows us to note that FSDP + LoRA not working is a known issue.

[danbider]

had a hard time pushing my commit through the spaces: here's my suggestion

Can I finetune using PEFT / LoRA?

• The LLM Foundry codebase does not directly have examples of PEFT or LORA workflows. However, our MPT model is a subclass of HuggingFace PretrainedModel, and Feature/peft compatible models #346 added required features to enable HuggingFace’s PEFT / LORA workflows for MPT. MPT models with LoRA modules can be trained either using LLM Foundry or Hugging Face's accelerate. Within LLM Foundry, run (scripts/train/train.py), adding lora arguments to the config .yaml, like so:

lora:
  args:
    r: 16
    lora_alpha: 32
    lora_dropout: 0.05
    target_modules: ['Wqkv']

• In the current release, these features have Beta support.
• For efficiency, The MPT model concatenates the Q, K, and V matrices in each attention block into a single Wqkv matrix that is three times wider. Currently, LoRA supports a low-rank approximation to this Wqkv matrix.
• Known issue: PEFT / LoRA do not directly work with FSDP.

[codestar12]

LGTM

[codestar12]

LGTM

[SeanTech99]

may I know how to use ddp config here in the yaml?

[germanjke]

@vchiley