Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

TGI: update to controller version 1.4.0 & bug fixes #470

Merged
merged 10 commits into from
Feb 8, 2024
Merged

TGI: update to controller version 1.4.0 & bug fixes #470

merged 10 commits into from
Feb 8, 2024

Commits on Feb 7, 2024

  1. feat(tgi): update controller version to 1.4.0

    @dacorvo
    dacorvo committed Feb 7, 2024
    Configuration menu
    Copy the full SHA
    7538703 View commit details
    Browse the repository at this point in the history

  2. fix(tgi): correctly handle single token inputs

    @dacorvo
    dacorvo committed Feb 7, 2024
    Configuration menu
    Copy the full SHA
    08ab3af View commit details
    Browse the repository at this point in the history

  3. feat(tgi): support seed generation parameter

    @dacorvo
    dacorvo committed Feb 7, 2024
    Configuration menu
    Copy the full SHA
    a647067 View commit details
    Browse the repository at this point in the history

Commits on Feb 8, 2024

  1. fix(tgi): return correct finish reason

    @dacorvo
    dacorvo committed Feb 8, 2024
    Configuration menu
    Copy the full SHA
    8dfdf75 View commit details
    Browse the repository at this point in the history

  2. fix(tgi): return only pending requests in next batch 

    On decode, finished requests were also included.
    @dacorvo
    dacorvo committed Feb 8, 2024
    Configuration menu
    Copy the full SHA
    28f9df6 View commit details
    Browse the repository at this point in the history

  3. fix(decoder): do not modify generation_config parameter

    @dacorvo
    dacorvo committed Feb 8, 2024
    Configuration menu
    Copy the full SHA
    468d129 View commit details
    Browse the repository at this point in the history

  4. fix(tgi): avoid repeated token in continuous batching 

    The last generated token of paused slots was recreated and sent back
instead of generating a new one.
    @dacorvo
    dacorvo committed Feb 8, 2024
    Configuration menu
    Copy the full SHA
    0727237 View commit details
    Browse the repository at this point in the history

  5. fix(tgi): update max_new_tokens in continuous batching 

    The max_new_tokens was not updated for pending requests while the
generated tokens were now actually seen as input tokens.
This was effectively as if the number of generated tokens had been reset.
    @dacorvo
    dacorvo committed Feb 8, 2024
    Configuration menu
    Copy the full SHA
    e034b14 View commit details
    Browse the repository at this point in the history

  6. test(tgi): add generator python tests

    @dacorvo
    dacorvo committed Feb 8, 2024
    Configuration menu
    Copy the full SHA
    e98974a View commit details
    Browse the repository at this point in the history

  7. test(tgi): add docker tests

    @dacorvo
    dacorvo committed Feb 8, 2024
    Configuration menu
    Copy the full SHA
    84c6877 View commit details
    Browse the repository at this point in the history