[Draft] Model library sidebar tab #837

mcmonkey4eva · 2024-09-15T07:50:45Z

Opening a draft PR for @huchenlei 's visibility and comments

This PR adds a sidebar tab to list and manage available models

Current State

it properly lists all main model folders, your personal folders within, and the models within
It does a nice lil "loading..." thing when you first click into a folder for which the model list isn't populated
Has a "(No Content)" for empty folders
displays the thumbnails of models next to the model
can hover models to get a popup showing some text info about the model
can click or drag a model to autospawn a loader node for the model
it uses the same tree system from the node library so it handles subfolders and wotnot just fine
Has code to compensate between windows & linux filepaths.
Has a setting for whether to load all model lists immediately or opening, or only load a list once you click a folder
Looks ugly.

TODO

Pretty it up, naturally
- figure out why the css is very subtly different from the other sidebar wtf aaaa
- ~~Load model thumbnail if available and render it (rather than just a lil file icon)~~
  - slightly hack css but works
- ~~maybe hide the .safetensors for cleanliness~~
  - Done
- ~~Add a preview view on hover, like the node preview, but showing the model's metadata~~
  - ugly but works
- add a way to actual interact with the metadata (eg copy the trigger phrase text to put in your prompt box)
~~Make it possible to drag the models out to nodes~~
- Done
- possibly a registry/store of x model type yields y node type? The core models are easy but needs an accessible storage API so custom nodes can register their own model types are handled in specific ways
  - Done
~~actually pull a proper list of folder types from upstream (currently hardcoded to the most common core types)~~
- Done
- arguably maybe just ask upstream for a dense model list rather than one folder at a time
- ~~(needs comfy API update either way)~~
- ~~waiting on Add route to list model types comfyanonymous/ComfyUI#4846~~

Maybe Todo Now Or Later

(Things that either can be in this PR or a followup)

Favoriting models you like
possibly a way to swap what node is used when dragging a model out for the user? eg there's a lot of custom lora loader nodes other than just the default one, so while the "initial MVP valid behavior" is just spawn a LoraLoader node, users will want ways to choose others
possibly a way to drag a model onto an existing node to swap the path in the node or something

Definitely Not Todo In This PR

Edit model data - that's more complex, should be a future followup
Move/rename/delete/etc. the model files - also can be a future followup
Metadata related things that require core comfy upgrades first:
- Filters on common metadata things (eg filter by model architecture class)
- search by text other than filename (eg search in description text)

in-progress

(incomplete list atm)

src/stores/modelStore.ts

src/components/sidebar/tabs/modelLibrary/ModelTreeLeaf.vue

huchenlei · 2024-09-15T08:10:54Z

src/App.vue

@@ -100,6 +101,14 @@ const init = () => {
    component: markRaw(NodeLibrarySidebarTab),
    type: 'vue'
  })
+  app.extensionManager.registerSidebarTab({
+    id: 'model-library',
+    icon: 'pi pi-folder-open',


We can probably pick another icon. Maybe pi-box?

yeah wasn't sure what icon would work here. Swapped to box

src/components/sidebar/tabs/ModelLibrarySidebarTab.vue

huchenlei · 2024-09-15T08:14:06Z

src/components/sidebar/tabs/ModelLibrarySidebarTab.vue

+    const children = node.children?.map(fillNodeInfo)
+    const model: ComfyModelDef | null =
+      node.leaf && node.data ? node.data : null
+    if (model && model.name.startsWith('\0')) {


Add a isLoading getter on ComfyModel should make logic here more explicit.

ngl every version of this code will be a hack other than directly adding the loading entry to the tree instead of passing a fake def, but that adds significant code complexity for the sake of dodging the hack.

But yeah I'll add a bool on the modeldef to at least mark it a bit more properly than this nullchar indicator

Not necessary adding a new field, you just need to add a getter that returns the expression model.name.startsWith('\0')

huchenlei · 2024-09-17T02:39:42Z

@mcmonkey4eva Gentle ping on the investigation of why calling api/models/{xxx} super slow.

mcmonkey4eva · 2024-09-17T04:31:41Z

In the current case I saw the other day and showed in that gif, I believe it may actually just be npm run dev has a rather slow proxy, the delay only happens on 5173 (dev server) but not on a direct call to 8188 (the comfy server). ie on actual deployed version it will not be so slow at all. (Though I have seen the comfy builtin webserver being quite slow to pickup calls in the past)

(well it will be slow if you have a lot of models but when I'm not on a laptop in tokyo I'll do some loadtesting and PR some aggro caching to comfy core, and maybe some fancy fastloading of file data too if python will let me get away with it)

based on node preview code

mcmonkey4eva · 2024-09-22T13:26:30Z

added a partial revert of 5a5a69d which was submitted directly to main without review and shockingly enough broke things

huchenlei · 2024-09-23T00:28:16Z

added a partial revert of 5a5a69d which was submitted directly to main without review and shockingly enough broke things

Can you explain why the revert is necessary?

mcmonkey4eva · 2024-09-23T09:04:08Z

surprise unreviewed commit broke the format in a way that doesn't make sense (text just yeets off the edge of the screen) and caused problems for this PR

For reference here's what that commit did that I reverted here:

the model library was even worse off than that

…ar-tab

mcmonkey4eva · 2024-09-23T09:46:14Z

Retargeted at dev1.3 per request in today's sync, seems to be working as intended locally

* basic/empty model library sidebar tab in-progress * make it actually list out models * extremely primitive search impl * list out available folders (incomplete list atm) * load list dynamically * nice lil loading icon * that's not doing anything * run autoformatter * fix up some absolute vue shenanigans * swap to pi-box * is_fake_object * i think apply the tailwind thingo * trim '.safetensors' from end of display title * oop * after load, retain title if no new title is given * is_load_requested to prevent duplication * dirty initial model metadata load & preview based on node preview code * update model store tests * initial image icon for model lib * i hate this * better empty spacer * add api handler for '/models' * load model folders list instead of hardcoding * add a 'no content' placeholder for empty folders * autoformat * autoload model metadata * error handling on metadata loading * larger model icons * click a model to spawn a node for it * draggable model nodes * add a setting for whether to autoload or not * autoformat will be the death of me * cleanup promise code * make the model preview actually half-decent * revert bad unchecked change * put registration back

807502278 · 2024-09-27T08:37:25Z

Excuse me, is the model library now finding the thumbnails and names of models through the Internet? I find that the names of many models are not the names of my local models, and can't read preview images with the same name as the model. Can a setting be added to switch between reading locally or obtaining from the Internet?

The authors of many model names write them casually. If so, I will change it, but it still has its original name in the model library.

mcmonkey4eva · 2024-09-28T05:31:34Z

@807502278 no it's not through the internet, it's the metadata in the header of the model file - so whatever title pops up is what the author typed in as the title at some point, and has been sitting inside your model the entire time you've had it.

Re showing the raw filename as an option instead of title, that's added in this pending PR: #1005

In the coming future we'll add tools to edit metadata freely, in the meantime you can use SwarmUI to edit the metadata (also the swarmui downloader can autoload better metadata from civit than what's baked into models by default if you give it a url)

807502278 · 2024-09-29T05:27:19Z

Thanks for telling me. It turns out to be the read model metadata. For each model I downloaded, the url and preview image are saved. I am very much looking forward to a tool that can freely edit metadata : )

mcmonkey4eva added 9 commits September 15, 2024 14:01

basic/empty model library sidebar tab

57b138d

in-progress

make it actually list out models

9df8e33

extremely primitive search impl

1a9e9b9

list out available folders

2f7ff2e

(incomplete list atm)

load list dynamically

486db95

nice lil loading icon

07cacec

that's not doing anything

6ffd685

run autoformatter

45fdb39

fix up some absolute vue shenanigans

d8c641c

huchenlei reviewed Sep 15, 2024

View reviewed changes

mcmonkey4eva added 3 commits September 15, 2024 17:59

swap to pi-box

773b4cc

is_fake_object

d0731d8

i think apply the tailwind thingo

c192818

mcmonkey4eva added 10 commits September 17, 2024 13:49

trim '.safetensors' from end of display title

4163af3

oop

6dd9491

after load, retain title if no new title is given

036bfc2

is_load_requested to prevent duplication

d9bae3b

dirty initial model metadata load & preview

d2cd56c

based on node preview code

Merge branch 'main' into model-library-sidebar-tab

2d2ad1c

update model store tests

9d9d776

initial image icon for model lib

f3a4e61

i hate this

9b3f8ab

better empty spacer

82ec4b2

mcmonkey4eva mentioned this pull request Sep 19, 2024

A model browser comfyanonymous/ComfyUI#4977

Closed

mcmonkey4eva added 4 commits September 21, 2024 15:15

Merge branch 'main' into model-library-sidebar-tab

e2a8878

add api handler for '/models'

b2ff7b4

load model folders list instead of hardcoding

600c087

add a 'no content' placeholder for empty folders

1ab37ef

mcmonkey4eva added 12 commits September 21, 2024 15:33

autoformat

2926b30

autoload model metadata

32c249b

error handling on metadata loading

b8ff73a

larger model icons

c72a0af

click a model to spawn a node for it

2c42e52

draggable model nodes

2b47da0

add a setting for whether to autoload or not

ab8ed33

autoformat will be the death of me

120dc7f

cleanup promise code

305cc82

make the model preview actually half-decent

a1a5e0d

Merge branch 'main' into model-library-sidebar-tab

3a6d179

revert bad unchecked change

2708688

mcmonkey4eva marked this pull request as ready for review September 22, 2024 13:26

mcmonkey4eva changed the base branch from main to dev1.3 September 23, 2024 09:39

mcmonkey4eva added 2 commits September 23, 2024 18:40

Merge remote-tracking branch 'origin/dev1.3' into model-library-sideb…

0375d42

…ar-tab

put registration back

7ebe10f

huchenlei merged commit 08ba155 into dev1.3 Sep 24, 2024
9 checks passed

huchenlei deleted the model-library-sidebar-tab branch September 24, 2024 00:48

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Draft] Model library sidebar tab #837

[Draft] Model library sidebar tab #837

mcmonkey4eva commented Sep 15, 2024 •

edited

Loading

huchenlei Sep 15, 2024

mcmonkey4eva Sep 15, 2024

huchenlei Sep 15, 2024

mcmonkey4eva Sep 15, 2024

huchenlei Sep 15, 2024

huchenlei commented Sep 17, 2024

mcmonkey4eva commented Sep 17, 2024 •

edited

Loading

mcmonkey4eva commented Sep 22, 2024

huchenlei commented Sep 23, 2024

mcmonkey4eva commented Sep 23, 2024 •

edited

Loading

mcmonkey4eva commented Sep 23, 2024

807502278 commented Sep 27, 2024 •

edited

Loading

mcmonkey4eva commented Sep 28, 2024 •

edited

Loading

807502278 commented Sep 29, 2024

[Draft] Model library sidebar tab #837

[Draft] Model library sidebar tab #837

Conversation

mcmonkey4eva commented Sep 15, 2024 • edited Loading

Current State

TODO

Maybe Todo Now Or Later

Definitely Not Todo In This PR

huchenlei Sep 15, 2024

Choose a reason for hiding this comment

mcmonkey4eva Sep 15, 2024

Choose a reason for hiding this comment

huchenlei Sep 15, 2024

Choose a reason for hiding this comment

mcmonkey4eva Sep 15, 2024

Choose a reason for hiding this comment

huchenlei Sep 15, 2024

Choose a reason for hiding this comment

huchenlei commented Sep 17, 2024

mcmonkey4eva commented Sep 17, 2024 • edited Loading

mcmonkey4eva commented Sep 22, 2024

huchenlei commented Sep 23, 2024

mcmonkey4eva commented Sep 23, 2024 • edited Loading

mcmonkey4eva commented Sep 23, 2024

807502278 commented Sep 27, 2024 • edited Loading

mcmonkey4eva commented Sep 28, 2024 • edited Loading

807502278 commented Sep 29, 2024

mcmonkey4eva commented Sep 15, 2024 •

edited

Loading

mcmonkey4eva commented Sep 17, 2024 •

edited

Loading

mcmonkey4eva commented Sep 23, 2024 •

edited

Loading

807502278 commented Sep 27, 2024 •

edited

Loading

mcmonkey4eva commented Sep 28, 2024 •

edited

Loading