You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
In #835, we made a change that results in the data loader being only initialized once per fit call, not once per iteration (i.e. epoch) as previously. Although we considered backwards compatibility, we missed one use case, namely passing a generator to iterator_train and iterator_valid (or, more precisely, a generator factory).
Now, if a user tries this, the training will run fine for the first epoch but afterwards the generator will be exhausted, resulting in the fit loop running 0 times, which creates strange results.
This issue should be addressed by two follow up tasks:
Re-enable passing a generator (factory) to iterator_train and iterator_valid. This can be achieved by checking the type and, if a types.GeneratorType is encountered, to wrap it into a class that makes it an iterator.
Place a guard inside the fit loop that detects if it is called 0 times. I would imagine this is never desired and should raise an error, or at the very least give a warning. As is, there is no error and no warning, but the results are very strange in the print log.
The text was updated successfully, but these errors were encountered:
As discussed with @ottonemo
In #835, we made a change that results in the data loader being only initialized once per fit call, not once per iteration (i.e. epoch) as previously. Although we considered backwards compatibility, we missed one use case, namely passing a generator to
iterator_train
anditerator_valid
(or, more precisely, a generator factory).Now, if a user tries this, the training will run fine for the first epoch but afterwards the generator will be exhausted, resulting in the fit loop running 0 times, which creates strange results.
This issue should be addressed by two follow up tasks:
iterator_train
anditerator_valid
. This can be achieved by checking the type and, if atypes.GeneratorType
is encountered, to wrap it into a class that makes it an iterator.The text was updated successfully, but these errors were encountered: