You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Has anyone tried to update this code for pytorch 0.4? awd-lstm-lm repo recently upgraded to make the models work in 0.4.
I made some attempts but got stuck at gradstat(), since in this method the code calls model.eval(), I encountered the following error while running with a trained model. See code here
Traceback (most recent call last):
File "dynamiceval.py", line 299, in <module>
gradstat(args, corpus, model, train_data, criterion)
File "dynamiceval.py", line 83, in gradstat
loss.backward()
File "/usr/anaconda3/lib/python3.6/site-packages/torch/tensor.py", line 93, in backward
torch.autograd.backward(self, gradient, retain_graph, create_graph)
File "/usr/anaconda3/lib/python3.6/site-packages/torch/autograd/__init__.py", line 89, in backward
allow_unreachable=True) # allow_unreachable flag
RuntimeError: backward_input can only be called in training mode
Looks like I cannot call loss.backward() when the model is set to eval mode. I've not tried running in with previous versions of pytorch. Is there a workaround for this in version 0.4?
Thanks.
The text was updated successfully, but these errors were encountered:
Thanks for raising this issue to my attention. Nice job getting so far, I looked into this myself and ran into the same problem, it appears that in pytorch 0.4 it is not possible to call loss.backward() on an RNN in eval mode, and we don't want to be using dropout so using train mode doesn't make sense. One hacky workaround that should work is to set all the dropout parameters of the model to 0 and use model.train() instead of model.eval(). I'm hoping to find a better solution though. I raised an issue about this in pytorch: pytorch/pytorch#10006 . I'll let you know when I have a 0.4 compatible version.
Hi there,
Has anyone tried to update this code for pytorch 0.4?
awd-lstm-lm
repo recently upgraded to make the models work in 0.4.I made some attempts but got stuck at
gradstat()
, since in this method the code callsmodel.eval()
, I encountered the following error while running with a trained model. See code hereLooks like I cannot call
loss.backward()
when the model is set toeval
mode. I've not tried running in with previous versions of pytorch. Is there a workaround for this in version0.4
?Thanks.
The text was updated successfully, but these errors were encountered: