This repository has been archived by the owner on May 28, 2024. It is now read-only.
Release 0.5.0 #111
Merged
Release 0.5.0 #111
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: Sihan Wang <[email protected]>
Signed-off-by: Sihan Wang <[email protected]>
Signed-off-by: Sihan Wang <[email protected]>
Signed-off-by: Sihan Wang <[email protected]>
Signed-off-by: Sihan Wang <[email protected]>
sihanwang41
requested review from
Yard1,
shrekris-anyscale and
akshay-anyscale
Co-authored-by: shrekris-anyscale <[email protected]> Signed-off-by: Sihan Wang <[email protected]>
Co-authored-by: shrekris-anyscale <[email protected]> Signed-off-by: Sihan Wang <[email protected]>
Co-authored-by: shrekris-anyscale <[email protected]> Signed-off-by: Sihan Wang <[email protected]>
Co-authored-by: shrekris-anyscale <[email protected]> Signed-off-by: Sihan Wang <[email protected]>
Co-authored-by: shrekris-anyscale <[email protected]> Signed-off-by: Sihan Wang <[email protected]>
Co-authored-by: shrekris-anyscale <[email protected]> Signed-off-by: Sihan Wang <[email protected]>
Co-authored-by: shrekris-anyscale <[email protected]> Signed-off-by: Sihan Wang <[email protected]>
Signed-off-by: Sihan Wang <[email protected]>
Signed-off-by: Sihan Wang <[email protected]>
Co-authored-by: shrekris-anyscale <[email protected]> Signed-off-by: Sihan Wang <[email protected]>
Signed-off-by: Sihan Wang <[email protected]>
… into 0.5.0_release2
shrekris-anyscale
approved these changes
Jan 8, 2024
|
How do i install tensorrt-llm in the ray-llm docker? I've tried to install tensorrt-llm several time but always stuck in the errors related to MPI or mpi4py |
Hi @rifkybujana, the trtllm will be installed by default inside the 0.5.0 image. Currently you can't install by yourself, we are planning to relax this restriction in the later version of rayllm. |
|
Hi @sihanwang41, thanks for the reply. When will the 0.5.0 image be released? Also which version of TensorRT-LLM would be installed? |
it will be out this week, 0.6.1 version will be installed. |
Signed-off-by: Sihan Wang <[email protected]>
Signed-off-by: Sihan Wang <[email protected]>
Yard1
approved these changes
Jan 18, 2024
avnishn
reviewed
Jan 18, 2024
| @@ -13,19 +11,17 @@ Jinja2 | |||
| numexpr>=2.7.3 | |||
| hf_transfer | |||
| evaluate | |||
| bitsandbytes | |||
| vllm>=0.2.0 | |||
There was a problem hiding this comment.
don't we need to pin this to <2.7?
alanwguo
pushed a commit
that referenced
this pull request
Jan 25, 2024
<img width="1253" alt="Screen Shot 2023-06-05 at 4 35 41 PM" src="https://github.com/anyscale/aviary/assets/20109646/9e71db45-dd3b-4fb8-88f8-2ec28a78ae6e"> Follow up: * Autoscaler: The current image is missing some dependencies for KubeRay autoscaling. * Frontend: The frontend cannot be launched directly due to some dependency issues (e.g. `gradio`, `pymongo`, `boto3`...). --------- Co-authored-by: Antoni Baum <[email protected]>
alanwguo
pushed a commit
that referenced
this pull request
Jan 25, 2024
Followup to #111 which adds a tutorial for GKE (the prior PR was for Amazon EKS). After all comments are addressed, before merging I will run through the tutorial again manually to make sure it works. - [ ] Final manual test "Fast follow" followups include: - [ ] Load test - [ ] Multiple models - [ ] Link to the tutorial from somewhere to make it discoverable. (Link to it from https://github.com/anyscale/aviary/blob/master/README.md I guess?) - [ ] Add a production guide using [RayService](https://ray-project.github.io/kuberay/guidance/rayservice/) More followups copied from #111: > Autoscaler: The current image is missing some dependencies for KubeRay autoscaling. > Frontend: The frontend cannot be launched directly due to some dependency issues (e.g. gradio, pymongo, boto3...). --------- Signed-off-by: Archit Kulkarni <[email protected]> Co-authored-by: Antoni Baum <[email protected]>
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.