Gradient disaster problem encountered during training

[gsynb]

Referring to the code usage examples, I think the usage methods of fastkan and efficient_kan should be consistent. I tried to replace MLP with KAN in my model. Using efficient_kan is normal, but using fastkan always causes the training loss to be nan. I haven't found the reason yet, so I'd like to ask if you have any solutions or insights?

[ZiyaoLi]

plz provide more information about the nan issue, such as creating the smallest code sample to reproduce the problem

[gsynb]

I tried to reproduce this problem with simple code, but was unsuccessful. I used this model https://github.com/QuantumLab-ZY/HamGNN And replaced the fully connected layer with KAN, however, eficientkan can operate normally, and fastkan keeps showing nan

[EladWarshawsky]

I have the very same issue

[EladWarshawsky]

I make a kan with the fastkanlinear layers rather than a mlp with linear layers and the loss comes out to a nan

[ZiyaoLi]

@EladWarshawsky @gsynb plz provide more input such as implementation and training curves for your question.