-
Notifications
You must be signed in to change notification settings - Fork 22
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merge main into documentation. #74
Commits on Jul 25, 2024
-
Adding image support to ollamaChat
* Refactor `messageHistory` to be agnostic of the image encoding. * Add backend encoding of images to `openAIChat`, `azureChat`, and `ollamaChat`. * Add image test points to the test files. Open question: Can we reliably detect which Ollama models support vision?
Configuration menu - View commit details
-
Copy full SHA for 8da8794 - Browse repository at this point
Copy the full SHA 8da8794View commit details -
Configuration menu - View commit details
-
Copy full SHA for 056c557 - Browse repository at this point
Copy the full SHA 056c557View commit details -
Configuration menu - View commit details
-
Copy full SHA for b9922f0 - Browse repository at this point
Copy the full SHA b9922f0View commit details -
Configuration menu - View commit details
-
Copy full SHA for ed2aa22 - Browse repository at this point
Copy the full SHA ed2aa22View commit details -
Configuration menu - View commit details
-
Copy full SHA for d24fa00 - Browse repository at this point
Copy the full SHA d24fa00View commit details
Commits on Jul 26, 2024
-
Configuration menu - View commit details
-
Copy full SHA for 1701188 - Browse repository at this point
Copy the full SHA 1701188View commit details -
Co-authored-by: MiriamScharnke <[email protected]>
Configuration menu - View commit details
-
Copy full SHA for bb4ba1a - Browse repository at this point
Copy the full SHA bb4ba1aView commit details -
Configuration menu - View commit details
-
Copy full SHA for 7aaadb4 - Browse repository at this point
Copy the full SHA 7aaadb4View commit details -
Update +llms/+utils/errorMessageCatalog.m
Co-authored-by: Christopher Creutzig <[email protected]>
Configuration menu - View commit details
-
Copy full SHA for 7bed93d - Browse repository at this point
Copy the full SHA 7bed93dView commit details -
Update +llms/+utils/errorMessageCatalog.m
Co-authored-by: Christopher Creutzig <[email protected]>
Configuration menu - View commit details
-
Copy full SHA for 83eff3a - Browse repository at this point
Copy the full SHA 83eff3aView commit details
Commits on Jul 29, 2024
-
Moved mlx into sub-directory, added Markdown export
Generating Markdown in pre-commit hook. The hook needs to be activated in the local repository, by including it in `.git/hooks/pre-commit`.
Configuration menu - View commit details
-
Copy full SHA for 1384590 - Browse repository at this point
Copy the full SHA 1384590View commit details -
Configuration menu - View commit details
-
Copy full SHA for c31e76e - Browse repository at this point
Copy the full SHA c31e76eView commit details -
Inserted text should have MATLAB®
Co-authored-by: MiriamScharnke <[email protected]>
Configuration menu - View commit details
-
Copy full SHA for 570aaa1 - Browse repository at this point
Copy the full SHA 570aaa1View commit details -
Configuration menu - View commit details
-
Copy full SHA for 50b9b0f - Browse repository at this point
Copy the full SHA 50b9b0fView commit details -
Future versions will get that automatically, from commit 570aaa1.
Configuration menu - View commit details
-
Copy full SHA for 581a4a2 - Browse repository at this point
Copy the full SHA 581a4a2View commit details -
Merge branch 'generate-md-from-mlx' of github.com:matlab-deep-learnin…
…g/llms-with-matlab into generate-md-from-mlx
Configuration menu - View commit details
-
Copy full SHA for 99c0902 - Browse repository at this point
Copy the full SHA 99c0902View commit details -
Merge pull request #59 from matlab-deep-learning/ollama-images
Ollama images
Configuration menu - View commit details
-
Copy full SHA for c3bec04 - Browse repository at this point
Copy the full SHA c3bec04View commit details
Commits on Jul 30, 2024
-
Merge pull request #62 from matlab-deep-learning/generate-md-from-mlx
Moved mlx into sub-directory, added Markdown export
Configuration menu - View commit details
-
Copy full SHA for ada0f7c - Browse repository at this point
Copy the full SHA ada0f7cView commit details -
Apply suggestions from code review
Co-authored-by: Deborah Ferreira <[email protected]>
Configuration menu - View commit details
-
Copy full SHA for 1ed0f7c - Browse repository at this point
Copy the full SHA 1ed0f7cView commit details -
Update doc/functions/openAIChat.md
Co-authored-by: Christopher Creutzig <[email protected]>
Configuration menu - View commit details
-
Copy full SHA for 595316f - Browse repository at this point
Copy the full SHA 595316fView commit details -
Configuration menu - View commit details
-
Copy full SHA for 8426bb7 - Browse repository at this point
Copy the full SHA 8426bb7View commit details
Commits on Jul 31, 2024
-
Merge pull request #60 from matlab-deep-learning/error-message-text-o…
…nly-edits Text only edits to error message catalog (doc review)
Configuration menu - View commit details
-
Copy full SHA for ddcf6f1 - Browse repository at this point
Copy the full SHA ddcf6f1View commit details
Commits on Aug 1, 2024
-
Configuration menu - View commit details
-
Copy full SHA for 72d171e - Browse repository at this point
Copy the full SHA 72d171eView commit details -
Delete doc/functions/openAIChat.md
This file was not meant for this change.
Configuration menu - View commit details
-
Copy full SHA for 223347b - Browse repository at this point
Copy the full SHA 223347bView commit details -
Merge pull request #66 from matlab-deep-learning/fix-openai-trademark
Fix openai trademark
Configuration menu - View commit details
-
Copy full SHA for ae07cd9 - Browse repository at this point
Copy the full SHA ae07cd9View commit details -
Configuration menu - View commit details
-
Copy full SHA for 57b22e1 - Browse repository at this point
Copy the full SHA 57b22e1View commit details
Commits on Aug 2, 2024
-
Merge pull request #67 from matlab-deep-learning/allow-char-StopSeque…
…nces Allow (long) char vectors for StopSequences
Configuration menu - View commit details
-
Copy full SHA for ea8874b - Browse repository at this point
Copy the full SHA ea8874bView commit details -
moondream is a much smaller vision model and perfectly suitable for the tests we want to run.
Configuration menu - View commit details
-
Copy full SHA for 172a4da - Browse repository at this point
Copy the full SHA 172a4daView commit details -
Configuration menu - View commit details
-
Copy full SHA for 5fa9534 - Browse repository at this point
Copy the full SHA 5fa9534View commit details -
Merge pull request #68 from matlab-deep-learning/replace-bakllava
Replace bakllava by moondream
Configuration menu - View commit details
-
Copy full SHA for e72cc99 - Browse repository at this point
Copy the full SHA e72cc99View commit details -
Merge pull request #69 from matlab-deep-learning/codeowners
Create CODEOWNERS
Configuration menu - View commit details
-
Copy full SHA for 062538c - Browse repository at this point
Copy the full SHA 062538cView commit details
Commits on Aug 5, 2024
-
Trace/replay
llms.internal.sendRequest
To decouple `texampleTests.m` from availability and speed of external servers, record calls to `llms.internal.sendRequest` (on dev machine) and replay (during most test runs, including CI). See tests/recording/README.md for instructions.
Configuration menu - View commit details
-
Copy full SHA for 2565f27 - Browse repository at this point
Copy the full SHA 2565f27View commit details -
Avoid bogus access to
json.choices.delta.content
The streaming interface sometimes responds with something like `json.choices = {"index":0,"delta":{},"logprobs":[],"finish_reason":"length"}`. Stop blindly assuming `json.choices.delta.content` exists.
Configuration menu - View commit details
-
Copy full SHA for 53c1df5 - Browse repository at this point
Copy the full SHA 53c1df5View commit details -
Merge pull request #71 from matlab-deep-learning/avoid-content-error
Avoid bogus access to `json.choices.delta.content`
Configuration menu - View commit details
-
Copy full SHA for fce8bb8 - Browse repository at this point
Copy the full SHA fce8bb8View commit details
Commits on Aug 6, 2024
-
Merge pull request #70 from matlab-deep-learning/test-doubles
Trace/replay `llms.internal.sendRequest`
Configuration menu - View commit details
-
Copy full SHA for 2cbda4d - Browse repository at this point
Copy the full SHA 2cbda4dView commit details -
The moondream model is unreliable in reporting peppers or even reporting anything at all, notice the empty responses in several of these calls: ``` >> disp(generate(chat,messages)) The image features a bunch of fruits and vegetables, including several bell peppers and chilies in various colors such as red, green, yellow, purple, and white. There is also an onion and some garlic on the table along with other food items that could be related to a meal or ingredient preparation. >> disp(generate(chat,messages)) The image displays a group of fresh vegetables on top of purple fabric. Among the vegetables are several peppers and garlic, some lying flat while others stand tall or in various positions. >> disp(generate(chat,messages)) urn of different colored vegetables such as peppers and onions are placed on a table. >> disp(generate(chat,messages)) The image shows an assortment of various colorful vegetables, primarily focused on the peppers. There is a wide range of vegetables displayed in different shapes and sizes, creating a visually appealing arrangement that highlights their unique characteristics. >> disp(generate(chat,messages)) >> disp(generate(chat,messages)) The image displays a table with an arrangement of colorful, whole vegetables such as peppers and onions. These ingredients are spread across the table, showcasing a variety of colors including reds, oranges, greens, yellows, and purples. There is also an onion among these vegetables, further adding to their vibrant appearance. Overall, the scene captures a visually appealing display of fresh produce. >> disp(generate(chat,messages)) >> disp(generate(chat,messages)) >> disp(generate(chat,messages)) The image shows a variety of brightly colored vegetables sitting on a table, including red bell pepper and green bell pepper. The vibrant hues and diverse shapes of the peppers create an eye-catching display against the purple background. >> disp(generate(chat,messages)) >> disp(generate(chat,messages)) xtracted image of many different kinds of vegetables including red bell peppers, onions and garlic, as well as carrots. >> disp(generate(chat,messages)) >> disp(generate(chat,messages)) The image features a variety of colorful vegetables, including bell peppers and garlic, all stacked up in an appealing manner. There are also several carrots mixed throughout the pile, contributing to the vibrant display. >> disp(generate(chat,messages)) Â The image features a pile of fresh vegetables, including numerous bell peppers in various shades. This vibrant collection includes both red and green varieties that create a visually appealing display on the table or surface they are placed on. >> disp(generate(chat,messages)) The image displays a variety of fresh vegetables, including bell peppers and carrots. ``` Since we are not interested in testing the model, but we do want to run an end-to-end test to ensure that we pass images in the data format required by Ollama, generate multiple responses and make sure that at least one of them mentions `"pepper"` or `"vegetable"`.
Configuration menu - View commit details
-
Copy full SHA for b5ad5d5 - Browse repository at this point
Copy the full SHA b5ad5d5View commit details -
Merge pull request #72 from matlab-deep-learning/img-descriptions
Allow "pepper" or "vegetable"
Configuration menu - View commit details
-
Copy full SHA for 150d9c1 - Browse repository at this point
Copy the full SHA 150d9c1View commit details