Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add Stable Diffusion ControlNet #359

Merged
merged 44 commits into from
Apr 8, 2024
Merged

Add Stable Diffusion ControlNet #359

Show file tree
Hide file tree
Changes from 26 commits
Commits
Show all changes
44 commits
Select commit Hold shift + click to select a range
1b62825
ControlNet setup
joelpaulkoch Feb 12, 2024
4dd11d8
Param mapping
joelpaulkoch Feb 14, 2024
abdea6b
AnyDoor controlnet params
joelpaulkoch Feb 15, 2024
c9ce1d2
No complaints from anydoor controlnet params
joelpaulkoch Feb 15, 2024
fa49ae9
Huggingface params
joelpaulkoch Feb 16, 2024
74fdcfb
unet with controlnet
joelpaulkoch Feb 17, 2024
8f55875
Define inputs for unet with controlnet
joelpaulkoch Feb 19, 2024
854ecd8
Unet with controlnet is same as in transformers
joelpaulkoch Feb 19, 2024
35b417f
StableDiffusionControlNet serving
joelpaulkoch Feb 19, 2024
99bf3f1
Merge branch 'main' into SD_controlnet
joelpaulkoch Feb 20, 2024
c109b1a
ControlNet input size
joelpaulkoch Feb 21, 2024
68957d1
Tuple as input for unet
joelpaulkoch Feb 23, 2024
211c209
Tuple as input inside stable diffusion
joelpaulkoch Feb 23, 2024
edfbd06
Determine conditioning size from spec
joelpaulkoch Feb 23, 2024
bfdd5a3
Add docs
joelpaulkoch Mar 4, 2024
95d0dff
Rename with_controlnet to with_additional_residuals
joelpaulkoch Mar 4, 2024
67b54d7
Update tests
joelpaulkoch Mar 4, 2024
8ab0b28
Update docs, rename with_controlnet to with_additional_residuals
joelpaulkoch Mar 4, 2024
32c2191
Compile with conditioning size
joelpaulkoch Mar 4, 2024
44e2b63
Preprocess image in stable diffusion task
joelpaulkoch Mar 4, 2024
061fcd4
Optional conditioning_scale input
joelpaulkoch Mar 13, 2024
de5dbcb
Test with tiny models
joelpaulkoch Mar 14, 2024
5007109
Rename test file to .exs
joelpaulkoch Mar 14, 2024
19ab327
Add optional additional residuals in :base architecture
joelpaulkoch Mar 14, 2024
0204cd6
Remove debug from test
joelpaulkoch Mar 14, 2024
7bfdfe7
Specify architecture
joelpaulkoch Mar 14, 2024
c7b9530
Remove residual templates
joelpaulkoch Apr 3, 2024
33ba7f8
Rename mid_block_residual to sample
joelpaulkoch Apr 3, 2024
1d1492f
Uncomment and update stable diffusion controlnet test
joelpaulkoch Apr 3, 2024
120b758
Update docs
joelpaulkoch Apr 5, 2024
a5ca864
Rename controlnet_conditioning to conditioning
joelpaulkoch Apr 5, 2024
b023fb8
Share optional add logic
joelpaulkoch Apr 5, 2024
83513d7
Rename and move to Bumblebee.Diffusion.ControlNet
joelpaulkoch Apr 5, 2024
2821a80
Fix mapping for layer names with trailing substitutions
jonatanklosko Apr 7, 2024
16bd3c2
Merge branch 'main' into SD_controlnet
jonatanklosko Apr 7, 2024
f002258
Wrap output in a container
jonatanklosko Apr 7, 2024
29c6f1d
Always infer conditioning size
jonatanklosko Apr 8, 2024
02382dd
Naming
jonatanklosko Apr 8, 2024
812cac3
Unify control_net -> controlnet
jonatanklosko Apr 8, 2024
5e6f755
Naming
jonatanklosko Apr 8, 2024
b8e0e54
Residuals -> states
jonatanklosko Apr 8, 2024
cd860de
Use tiny checkpoint and reference outputs in tests
jonatanklosko Apr 8, 2024
361926c
Update docs sidebar
jonatanklosko Apr 8, 2024
cba1501
Up
jonatanklosko Apr 8, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions lib/bumblebee.ex
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -116,6 +116,7 @@ defmodule Bumblebee do
"CLIPModel" => {Bumblebee.Multimodal.Clip, :base},
"CLIPTextModel" => {Bumblebee.Text.ClipText, :base},
"CLIPVisionModel" => {Bumblebee.Vision.ClipVision, :base},
"ControlNetModel" => {Bumblebee.Diffusion.StableDiffusion.ControlNet, :base},
"ConvNextForImageClassification" => {Bumblebee.Vision.ConvNext, :for_image_classification},
"ConvNextModel" => {Bumblebee.Vision.ConvNext, :base},
"DeiTForImageClassification" => {Bumblebee.Vision.Deit, :for_image_classification},
Expand Down
Loading
Loading