NaN value while training

[louiseblade]

I've read google team implementation too, and they use gamma normalization to prevent NaN while training .
when i use your code, first several epochs were fine but suddenly torch.autograd detect anomaly while training . I do believe that's because of the RGLRU . Is there anyway to avoid NaN or should we use some sort of normalization ?

[proger]

Hey, could you share where that nan comes from? The anomaly detector should be helping with that.

[louiseblade]

Thank you for you reply . I'm using pycharm and here is the log