Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merge main into documentation. #74

Merged
merged 37 commits into from
Aug 9, 2024
Merged

Merge main into documentation. #74

merged 37 commits into from
Aug 9, 2024

Commits on Jul 25, 2024

  1. Adding image support to ollamaChat

    * Refactor `messageHistory` to be agnostic of the image encoding.
    * Add backend encoding of images to `openAIChat`, `azureChat`, and `ollamaChat`.
    * Add image test points to the test files.
    
    Open question: Can we reliably detect which Ollama models support vision?
    ccreutzi committed Jul 25, 2024
    Configuration menu
    Copy the full SHA
    8da8794 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    056c557 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    b9922f0 View commit details
    Browse the repository at this point in the history
  4. Preinstall bakllava

    Preinstall bakllava, to have a vision model
    ccreutzi committed Jul 25, 2024
    Configuration menu
    Copy the full SHA
    ed2aa22 View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    d24fa00 View commit details
    Browse the repository at this point in the history

Commits on Jul 26, 2024

  1. Configuration menu
    Copy the full SHA
    1701188 View commit details
    Browse the repository at this point in the history
  2. Update doc/Ollama.md

    Co-authored-by: MiriamScharnke <[email protected]>
    ccreutzi and MiriamScharnke committed Jul 26, 2024
    Configuration menu
    Copy the full SHA
    bb4ba1a View commit details
    Browse the repository at this point in the history
  3. WARNING -> TIP

    ccreutzi committed Jul 26, 2024
    Configuration menu
    Copy the full SHA
    7aaadb4 View commit details
    Browse the repository at this point in the history
  4. Update +llms/+utils/errorMessageCatalog.m

    Co-authored-by: Christopher Creutzig <[email protected]>
    MiriamScharnke and ccreutzi committed Jul 26, 2024
    Configuration menu
    Copy the full SHA
    7bed93d View commit details
    Browse the repository at this point in the history
  5. Update +llms/+utils/errorMessageCatalog.m

    Co-authored-by: Christopher Creutzig <[email protected]>
    MiriamScharnke and ccreutzi committed Jul 26, 2024
    Configuration menu
    Copy the full SHA
    83eff3a View commit details
    Browse the repository at this point in the history

Commits on Jul 29, 2024

  1. Moved mlx into sub-directory, added Markdown export

    Generating Markdown in pre-commit hook.
    
    The hook needs to be activated in the local repository, by including it in `.git/hooks/pre-commit`.
    ccreutzi committed Jul 29, 2024
    Configuration menu
    Copy the full SHA
    1384590 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    c31e76e View commit details
    Browse the repository at this point in the history
  3. Inserted text should have MATLAB®

    Co-authored-by: MiriamScharnke <[email protected]>
    ccreutzi and MiriamScharnke committed Jul 29, 2024
    Configuration menu
    Copy the full SHA
    570aaa1 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    50b9b0f View commit details
    Browse the repository at this point in the history
  5. Add ® in generated files

    Future versions will get that automatically, from commit 570aaa1.
    ccreutzi committed Jul 29, 2024
    Configuration menu
    Copy the full SHA
    581a4a2 View commit details
    Browse the repository at this point in the history
  6. Merge branch 'generate-md-from-mlx' of github.com:matlab-deep-learnin…

    …g/llms-with-matlab into generate-md-from-mlx
    ccreutzi committed Jul 29, 2024
    Configuration menu
    Copy the full SHA
    99c0902 View commit details
    Browse the repository at this point in the history
  7. Configuration menu
    Copy the full SHA
    c3bec04 View commit details
    Browse the repository at this point in the history

Commits on Jul 30, 2024

  1. Merge pull request #62 from matlab-deep-learning/generate-md-from-mlx

    Moved mlx into sub-directory, added Markdown export
    ccreutzi committed Jul 30, 2024
    Configuration menu
    Copy the full SHA
    ada0f7c View commit details
    Browse the repository at this point in the history
  2. Apply suggestions from code review

    Co-authored-by: Deborah Ferreira <[email protected]>
    MiriamScharnke and debymf committed Jul 30, 2024
    Configuration menu
    Copy the full SHA
    1ed0f7c View commit details
    Browse the repository at this point in the history
  3. Update doc/functions/openAIChat.md

    Co-authored-by: Christopher Creutzig <[email protected]>
    MiriamScharnke and ccreutzi committed Jul 30, 2024
    Configuration menu
    Copy the full SHA
    595316f View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    8426bb7 View commit details
    Browse the repository at this point in the history

Commits on Jul 31, 2024

  1. Merge pull request #60 from matlab-deep-learning/error-message-text-o…

    …nly-edits
    
    Text only edits to error message catalog (doc review)
    MiriamScharnke committed Jul 31, 2024
    Configuration menu
    Copy the full SHA
    ddcf6f1 View commit details
    Browse the repository at this point in the history

Commits on Aug 1, 2024

  1. Fix OpenAI trademarks

    MiriamScharnke committed Aug 1, 2024
    Configuration menu
    Copy the full SHA
    72d171e View commit details
    Browse the repository at this point in the history
  2. Delete doc/functions/openAIChat.md

    This file was not meant for this change.
    MiriamScharnke committed Aug 1, 2024
    Configuration menu
    Copy the full SHA
    223347b View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    ae07cd9 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    57b22e1 View commit details
    Browse the repository at this point in the history

Commits on Aug 2, 2024

  1. Merge pull request #67 from matlab-deep-learning/allow-char-StopSeque…

    …nces
    
    Allow (long) char vectors for StopSequences
    ccreutzi committed Aug 2, 2024
    Configuration menu
    Copy the full SHA
    ea8874b View commit details
    Browse the repository at this point in the history
  2. Replace bakllava by moondream

    moondream is a much smaller vision model and perfectly suitable for the tests we want to run.
    ccreutzi committed Aug 2, 2024
    Configuration menu
    Copy the full SHA
    172a4da View commit details
    Browse the repository at this point in the history
  3. Create CODEOWNERS

    ccreutzi committed Aug 2, 2024
    Configuration menu
    Copy the full SHA
    5fa9534 View commit details
    Browse the repository at this point in the history
  4. Merge pull request #68 from matlab-deep-learning/replace-bakllava

    Replace bakllava by moondream
    ccreutzi committed Aug 2, 2024
    Configuration menu
    Copy the full SHA
    e72cc99 View commit details
    Browse the repository at this point in the history
  5. Merge pull request #69 from matlab-deep-learning/codeowners

    Create CODEOWNERS
    ccreutzi committed Aug 2, 2024
    Configuration menu
    Copy the full SHA
    062538c View commit details
    Browse the repository at this point in the history

Commits on Aug 5, 2024

  1. Trace/replay llms.internal.sendRequest

    To decouple `texampleTests.m` from availability and speed of external servers,
    record calls to `llms.internal.sendRequest` (on dev machine) and replay
    (during most test runs, including CI). See tests/recording/README.md for
    instructions.
    ccreutzi committed Aug 5, 2024
    Configuration menu
    Copy the full SHA
    2565f27 View commit details
    Browse the repository at this point in the history
  2. Avoid bogus access to json.choices.delta.content

    The streaming interface sometimes responds with something like `json.choices = {"index":0,"delta":{},"logprobs":[],"finish_reason":"length"}`. Stop blindly assuming `json.choices.delta.content` exists.
    ccreutzi committed Aug 5, 2024
    Configuration menu
    Copy the full SHA
    53c1df5 View commit details
    Browse the repository at this point in the history
  3. Merge pull request #71 from matlab-deep-learning/avoid-content-error

    Avoid bogus access to `json.choices.delta.content`
    ccreutzi committed Aug 5, 2024
    Configuration menu
    Copy the full SHA
    fce8bb8 View commit details
    Browse the repository at this point in the history

Commits on Aug 6, 2024

  1. Merge pull request #70 from matlab-deep-learning/test-doubles

    Trace/replay `llms.internal.sendRequest`
    ccreutzi committed Aug 6, 2024
    Configuration menu
    Copy the full SHA
    2cbda4d View commit details
    Browse the repository at this point in the history
  2. Allow "pepper" or "vegetable"

    The moondream model is unreliable in reporting peppers or even reporting anything at all, notice the empty responses in several of these calls:
    
    ```
    >> disp(generate(chat,messages))
    
     The image features a bunch of fruits and vegetables, including several bell peppers and chilies in various colors such as red, green, yellow, purple, and white. There is also an onion and some garlic on the table along with other food items that could be related to a meal or ingredient preparation.
    >> disp(generate(chat,messages))
    
     The image displays a group of fresh vegetables on top of purple fabric. Among the vegetables are several peppers and garlic, some lying flat while others stand tall or in various positions.
    >> disp(generate(chat,messages))
    urn of different colored vegetables such as peppers and onions are placed on a table.
    >> disp(generate(chat,messages))
    
     The image shows an assortment of various colorful vegetables, primarily focused on the peppers. There is a wide range of vegetables displayed in different shapes and sizes, creating a visually appealing arrangement that highlights their unique characteristics.
    >> disp(generate(chat,messages))
    >> disp(generate(chat,messages))
    
     The image displays a table with an arrangement of colorful, whole vegetables such as peppers and onions. These ingredients are spread across the table, showcasing a variety of colors including reds, oranges, greens, yellows, and purples. There is also an onion among these vegetables, further adding to their vibrant appearance. Overall, the scene captures a visually appealing display of fresh produce.
    >> disp(generate(chat,messages))
    >> disp(generate(chat,messages))
    >> disp(generate(chat,messages))
     The image shows a variety of brightly colored vegetables sitting on a table, including red bell pepper and green bell pepper. The vibrant hues and diverse shapes of the peppers create an eye-catching display against the purple background.
    >> disp(generate(chat,messages))
    >> disp(generate(chat,messages))
    xtracted image of many different kinds of vegetables including red bell peppers, onions and garlic, as well as carrots.
    >> disp(generate(chat,messages))
    >> disp(generate(chat,messages))
    
    The image features a variety of colorful vegetables, including bell peppers and garlic, all stacked up in an appealing manner. There are also several carrots mixed throughout the pile, contributing to the vibrant display.
    >> disp(generate(chat,messages))
    Â The image features a pile of fresh vegetables, including numerous bell peppers in various shades. This vibrant collection includes both red and green varieties that create a visually appealing display on the table or surface they are placed on.
    >> disp(generate(chat,messages))
    
     The image displays a variety of fresh vegetables, including bell peppers and carrots.
    ```
    
    Since we are not interested in testing the model, but we do want to run an end-to-end test to ensure that we pass images in the data format required by Ollama, generate multiple responses and make sure that at least one of them mentions `"pepper"` or `"vegetable"`.
    ccreutzi committed Aug 6, 2024
    Configuration menu
    Copy the full SHA
    b5ad5d5 View commit details
    Browse the repository at this point in the history
  3. Merge pull request #72 from matlab-deep-learning/img-descriptions

    Allow "pepper" or "vegetable"
    ccreutzi committed Aug 6, 2024
    Configuration menu
    Copy the full SHA
    150d9c1 View commit details
    Browse the repository at this point in the history