Skip to content

Commit

Permalink
fix(llama.cpp): disable mirostat as default
Browse files Browse the repository at this point in the history
Even if increasing the quality of the output, it has shown to have
performance drawbacks to be so noticeable that the confuses users about
speed of LocalAI ( see also
#2780 ).

This changeset disables Mirostat by default (which can
be still enabled manually).

Signed-off-by: Ettore Di Giacinto <[email protected]>
  • Loading branch information
mudler committed Jul 18, 2024
1 parent 96f67ef commit 4e40788
Showing 1 changed file with 2 additions and 1 deletion.
3 changes: 2 additions & 1 deletion core/config/backend_config.go
Original file line number Diff line number Diff line change
Expand Up @@ -261,7 +261,8 @@ func (cfg *BackendConfig) SetDefaults(opts ...ConfigLoaderOption) {
defaultTopP := 0.95
defaultTopK := 40
defaultTemp := 0.9
defaultMirostat := 2
// https://github.com/mudler/LocalAI/issues/2780
defaultMirostat := 0
defaultMirostatTAU := 5.0
defaultMirostatETA := 0.1
defaultTypicalP := 1.0
Expand Down

0 comments on commit 4e40788

Please sign in to comment.