very simple strong-cache on model list

[mcmonkey4eva]

This is based on the same work as #4968 but a much simpler and more agreeable approach: when object_info is hit, use a strong cache to ensure each model list is read exactly once, rather than once per node - ie skip the cache validation within the current call, until the next object_info hit.

Currently only /object_info triggers a cache clear, but it can be made to apply to /object_info/{x} and /models/{x} as well if desired. (I'd argue that strong caching in those subcalls is desired, as only /object_info is intended as a 'refresh' - though also a separate dedicated /refresh route would be preferred imo)

For this code I made a nice lil subclass helper doodle to reduce the amount of chaotic global variable pollution in that file a bit.

The practical result is, in the ultra-high-latency test env I set up, time to read /object_info using just this cache PR alone reduces from 70 seconds to 23 seconds.

The cache hit/miss pattern of a pure default comfy with no custom nodes for /object_info is:

cache hit for controlnet
cache miss for style_models
cache hit for style_models
cache miss for clip_vision
cache hit for clip_vision
cache hit for checkpoints
cache hit for checkpoints
cache miss for gligen
cache hit for gligen
cache miss for configs
cache hit for checkpoints
cache hit for configs
cache hit for checkpoints
cache hit for loras
cache hit for loras
cache miss for hypernetworks
cache hit for hypernetworks
cache miss for upscale_models
cache hit for upscale_models
cache hit for checkpoints
cache hit for checkpoints
cache miss for photomaker
cache hit for photomaker
cache hit for clip
cache hit for clip
cache hit for clip
cache hit for clip
cache hit for controlnet
cache miss for style_models
cache hit for style_models
cache miss for clip_vision
cache hit for clip_vision
cache hit for checkpoints
cache hit for checkpoints
cache miss for gligen
cache hit for gligen
cache miss for configs
cache hit for checkpoints
cache hit for configs
cache hit for checkpoints
cache hit for loras
cache hit for loras
cache miss for hypernetworks
cache hit for hypernetworks
cache miss for upscale_models
cache hit for upscale_models
cache hit for checkpoints
cache hit for checkpoints
cache miss for photomaker
cache hit for loras
cache miss for hypernetworks
cache hit for hypernetworks
cache miss for upscale_models
cache hit for upscale_models
cache miss for upscale_models
cache hit for upscale_models
cache hit for checkpoints
cache hit for checkpoints
cache miss for photomaker
cache hit for photomaker
cache hit for clip
cache hit for clip
cache hit for clip
cache hit for clip
cache hit for clip
cache hit for clip

[comfyanonymous]

This might break situations where ComfyUI is run in api only mode where people might be adding new models to the folders without doing a object_info call

[mcmonkey4eva]

I've changed it temporarily to only cache within the /object_info call and skip the strong cache for all other cases. This makes the core functional intent still work while preventing any side effects. Should be happily mergeable now.

In the long run, I think the solution is a /refresh API route and give API developers notice for a month or two that the route should be used when changing files around, and then after the notice period enable a very strong global cache - even eg object_info's entire block should be strongly cached and only cleared when /refresh is hit. Everything should be in memory, not live scanning the disk all day, except exactly when refresh is called to check the disk

[comfyanonymous]

This would look cleaner if it was using a context.

[mcmonkey4eva]

good point - applied