Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

improve text-to-image task page #889

Merged
merged 32 commits into from
Sep 9, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
32 commits
Select commit Hold shift + click to select a range
6e0a3bc
little change to task description
linoytsaban Sep 3, 2024
cbb3969
Merge branch 'huggingface:main' into text2img-task
linoytsaban Sep 3, 2024
0858b96
add initial task variants
linoytsaban Sep 3, 2024
23d9d44
Merge branch 'main' into text2img-task
linoytsaban Sep 4, 2024
017ff04
add image editing task
linoytsaban Sep 4, 2024
baa2c37
add personalization
linoytsaban Sep 5, 2024
d9ef465
add links
linoytsaban Sep 5, 2024
6ab90ed
Merge remote-tracking branch 'origin/text2img-task' into text2img-task
linoytsaban Sep 5, 2024
19463e4
Merge branch 'main' into text2img-task
linoytsaban Sep 5, 2024
47801f2
format
linoytsaban Sep 5, 2024
b5f8250
format
linoytsaban Sep 5, 2024
2c8b759
format
linoytsaban Sep 5, 2024
812f8e2
format
linoytsaban Sep 5, 2024
8aa6daa
Merge branch 'main' into text2img-task
linoytsaban Sep 6, 2024
9b792c1
Update packages/tasks/src/tasks/text-to-image/about.md
linoytsaban Sep 6, 2024
39bdbe1
Update packages/tasks/src/tasks/text-to-image/about.md
linoytsaban Sep 6, 2024
6a71578
simplify real image editing
linoytsaban Sep 6, 2024
532c779
Merge remote-tracking branch 'origin/text2img-task' into text2img-task
linoytsaban Sep 6, 2024
8c292c3
format
linoytsaban Sep 6, 2024
e15a303
format
linoytsaban Sep 6, 2024
36a1e3d
Merge branch 'main' into text2img-task
linoytsaban Sep 6, 2024
1f9d0af
Update packages/tasks/src/tasks/text-to-image/data.ts
linoytsaban Sep 7, 2024
dbbfd4f
changes to personalization variant
linoytsaban Sep 7, 2024
8bc84cc
fix images
linoytsaban Sep 7, 2024
4d7fef7
Merge branch 'main' into text2img-task
linoytsaban Sep 7, 2024
9754f20
format
linoytsaban Sep 7, 2024
1a6c542
Merge remote-tracking branch 'origin/text2img-task' into text2img-task
linoytsaban Sep 7, 2024
4be7f12
change to image editing description
linoytsaban Sep 8, 2024
a858026
add figure refs
linoytsaban Sep 8, 2024
144c4ca
format
linoytsaban Sep 8, 2024
43c297c
format
linoytsaban Sep 8, 2024
9c306b6
add back missing title
linoytsaban Sep 8, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 24 additions & 3 deletions packages/tasks/src/tasks/text-to-image/about.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

### Data Generation

Businesses can generate data for their their use cases by inputting text and getting image outputs.
Businesses can generate data for their use cases by inputting text and getting image outputs.

### Immersive Conversational Chatbots

Expand All @@ -16,9 +16,27 @@ Different patterns can be generated to obtain unique pieces of fashion. Text-to-

Architects can utilise the models to construct an environment based out on the requirements of the floor plan. This can also include the furniture that has to be placed in that environment.

## Task Variants
## Task Variants

You can contribute variants of this task [here](https://github.com/huggingface/hub-docs/blob/main/tasks/src/text-to-image/about.md).
### Image Editing

Image editing with text-to-image models involves modifying an image following edit instructions provided in a text prompt.

- **Synthetic image editing**: Adjusting images that were initially created using an input prompt while preserving the overall meaning or context of the original image.

![Examples](https://huggingface.co/datasets/diffusers/diffusers-images-docs/resolve/main/edit_p2p.png)
_Figure taken from ["InstructPix2Pix: Learning to Follow Image Editing Instructions"](https://www.timothybrooks.com/instruct-pix2pix)_

- **Real image editing**: Similar to synthetic image editing, except we're using real photos/images. This task is usually more complex.

![Examples](https://huggingface.co/datasets/diffusers/diffusers-images-docs/resolve/main/pix2pix.jpeg)
_Figure taken from ["Prompt-to-Prompt Image Editing with Cross-Attention Control"](https://prompt-to-prompt.github.io)_

### Personalization

Personalization refers to techniques used to customize text-to-image models. We introduce new subjects or concepts to the model, which the model can then generate when we refer to them with a text prompt.

For example, you can use these techniques to generate images of your dog in imaginary settings, after you have taught the model using a few reference images of the subject (or just one in some cases). Teaching the model a new concept can be achieved through fine-tuning, or by using training-free techniques.

## Inference

Expand Down Expand Up @@ -65,11 +83,14 @@ await inference.textToImage({
- [Introducing Würstchen: Fast Diffusion for Image Generation](https://huggingface.co/blog/wuerstchen)
- [Efficient Controllable Generation for SDXL with T2I-Adapters](https://huggingface.co/blog/t2i-sdxl-adapters)
- [Welcome aMUSEd: Efficient Text-to-Image Generation](https://huggingface.co/blog/amused)
- Image Editing Demos: [LEDITS++](https://huggingface.co/spaces/editing-images/leditsplusplus), [Turbo Edit](https://huggingface.co/spaces/turboedit/turbo_edit), [InstructPix2Pix](https://huggingface.co/spaces/timbrooks/instruct-pix2pix), [CosXL](https://huggingface.co/spaces/multimodalart/cosxl)
- Training free Personalization Demos: [Face-to-All](https://huggingface.co/spaces/multimodalart/face-to-all), [InstantStyle](https://huggingface.co/spaces/InstantX/InstantStyle), [RB-modulation](https://huggingface.co/spaces/fffiloni/RB-Modulation), [Photomaker v2](https://huggingface.co/spaces/TencentARC/PhotoMaker-V2)

### Model Fine-tuning

- [Finetune Stable Diffusion Models with DDPO via TRL](https://huggingface.co/blog/pref-tuning)
- [LoRA training scripts of the world, unite!](https://huggingface.co/blog/sdxl_lora_advanced_script)
- [Using LoRA for Efficient Stable Diffusion Fine-Tuning](https://huggingface.co/blog/lora)
- LoRA fine tuning Spaces: [FLUX.1 finetuning](https://huggingface.co/spaces/autotrain-projects/train-flux-lora-ease), [SDXL finetuning](https://huggingface.co/spaces/multimodalart/lora-ease)

This page was made possible thanks to the efforts of [Ishan Dutta](https://huggingface.co/ishandutta), [Enrique Elias Ubaldo](https://huggingface.co/herrius) and [Oğuz Akif](https://huggingface.co/oguzakif).
2 changes: 1 addition & 1 deletion packages/tasks/src/tasks/text-to-image/data.ts
Original file line number Diff line number Diff line change
Expand Up @@ -92,7 +92,7 @@ const taskData: TaskDataCustom = {
},
],
summary:
"Generates images from input text. These models can be used to generate and modify images based on text prompts.",
"Text-to-image is the task of generating images from input text. These pipelines can also be used to modify and edit images based on text prompts.",
widgetModels: ["black-forest-labs/FLUX.1-dev"],
youtubeId: "",
};
Expand Down
Loading