Non-closure Prodigy?

[madman404]

I've gotten a lot of use out of prodigy over the past few months, and I'd love if I could take advantage of schedule-free optimization alongside it. I see that there is a reference example in the repo, but it uses closure. The problem is that the training loop I am using is not set up for closure and I am not very smart, and I don't understand nearly enough about the math here to create a non-closure implementation. Would it be possible to provide one like what has been provided for AdamW and SGD?

[adefazio]

We are working on an implementation, we are just finalizing the theory now. We will need to verify that it works correctly on a large set of problems before we release it.

[sdbds]

I'm waiting for this too.

[madman404]

Is the new-ish wrapper compatible with prodigy, or is there still work being done on a dedicated combination of the two? I understand the wrapper is meant for arbitrary optimizers, but prodigy is also a little more... involved than usual.

[adefazio]

I've been experimenting with combining the two but I'm not seeing performance on par with using Prodigy by it's self yet. I'm still unsure why.