Skip to content

Commit

Permalink
change docstring (#563)
Browse files Browse the repository at this point in the history
  • Loading branch information
bmosaicml committed Aug 28, 2023
1 parent 7f373e3 commit aad9f64
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion llmfoundry/optim/adaptive_lion.py
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ class DecoupledAdaLRLion(Optimizer):
simple windowed moving average (MVA) of moment norms taken from steps T-1000
to T-500. If an outlier is detected, the LR is lowered by `lr_penalty` for
`timeout` steps. If N outliers are detected within `timeout` steps, the LR
is scaled down by min(`lr_penalty` ** N, `min_scale`).
is scaled down by max(`lr_penalty` ** N, `min_scale`).
Args:
params (Iterable[torch.Parameter]): Model parameters to optimize
Expand Down

0 comments on commit aad9f64

Please sign in to comment.