⚡Add Support for LoRA Training on Flux-Dev2Pro Models

[sashaok123]

Description:
The Flux-Dev2Pro model is a fine-tuned version of Flux-dev, designed to overcome some of the challenges seen when training LoRA models on Flux-dev.
However, currently, the sd-scripts do not support Flux-Dev2Pro as a base for LoRA training.
This feature would greatly enhance LoRA training quality and help avoid common pitfalls like model collapse and distorted outputs.

Problem:

1. Flux-Dev2Pro is a fine-tuned model designed to address the lack of distillation guidance in the original Flux-dev. This model offers a more stable foundation for LoRA training but requires specific training conditions, such as setting the guidance scale to 1.0.
2. The current version of sd-scripts does not include support or specific adaptations for Flux-Dev2Pro, which means users cannot benefit from this more robust model for LoRA training.
3. Without these adjustments, training on Flux-Dev2Pro leads to issues like distorted human limbs, poor semantic understanding, and model collapse when using standard settings.

Proposed Solution:

1. Add Flux-Dev2Pro support to the sd-scripts repository, including specific adaptations for this model during LoRA training.
2. Ensure the scripts default to a guidance scale of 1.0 when Flux-Dev2Pro is used, as this helps minimize distillation loss and better replicates the conditions under which Flux-pro was trained.
3. Introduce an option to select Flux-Dev2Pro as a base model for training LoRA, with appropriate modifications to the training dynamics to prevent guidance embedding issues.
   

Examples:

In experiments described in the article linked below, LoRA training on Flux-Dev2Pro and applying the results to Flux-dev yielded superior performance compared to other methods. Training on Flux-Dev2Pro avoids the collapse and instability seen with Flux-dev while producing significantly higher-quality results.

Request:

Could you update the sd-scripts to:

1. Add Flux-Dev2Pro as a recognized base model for LoRA training.
2. Implement the necessary training parameter adjustments, specifically setting the guidance scale to 1.0 for this model.

Download Link:
You can download Flux-Dev2Pro from Hugging Face:
https://huggingface.co/ashen0209/Flux-Dev2Pro

These changes would enhance LoRA training quality and offer users access to a more stable fine-tuned model, ultimately improving the outcomes of text-to-image generation.

Source:
Why Flux LoRA So Hard to Train and How to Overcome It

[kohya-ss]

It seems that sdbds has converted the model to Black Forest Labs' safetensors format, which allows us to train LoRA directly using sd-scripts. https://huggingface.co/bdsqlsz/flux1-dev2pro-single