Skip to content

Commit

Permalink
docs: update TGI container
Browse files Browse the repository at this point in the history
  • Loading branch information
dacorvo committed Apr 5, 2024
1 parent 7dfd2c6 commit fa21917
Showing 1 changed file with 4 additions and 4 deletions.
8 changes: 4 additions & 4 deletions docs/source/containers.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -10,20 +10,20 @@ an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express o
specific language governing permissions and limitations under the License.
-->

# Optimum Neuron Container
# Optimum Neuron Container

We provide pre-built Optimum Neuron containers for Amazon SageMaker. These containers come with all of the Hugging Face libraries and dependencies pre-installed, so you can start using them right away.
We have containers for training and inference, and optimized text generation containers with TGI. The table is up to date and only includes the latest versions of each container. You can find older versions in the [Deep Learning Container Release Notes](https://github.com/aws/deep-learning-containers/releases?q=hf-neuronx&expanded=true)

We recommend using the `sagemaker` Python SDK to retrieve the image URI for the container you want to use.
We recommend using the `sagemaker` Python SDK to retrieve the image URI for the container you want to use.

## Available Optimum Neuron Containers

| Type | Optimum Version | Image URI |
|-----------------------------|-----------------|---------------------------------------------|
| Training | 0.0.13 | `763104351884.dkr.ecr.us-west-2.amazonaws.com/huggingface-pytorch-training-neuronx:1.13.1-neuronx-py310-sdk2.15.0-ubuntu20.04` |
| Inference | 0.0.13 | `763104351884.dkr.ecr.us-west-2.amazonaws.com/huggingface-pytorch-inference-neuronx:1.13.1-neuronx-py310-sdk2.15.0-ubuntu20.04` |
| Text Generation Inference | 0.0.17 | `763104351884.dkr.ecr.us-west-2.amazonaws.com/huggingface-pytorch-tgi-inference:1.13.1-optimum0.0.17-neuronx-py310-ubuntu22.04` |
| Text Generation Inference | 0.0.20 | `763104351884.dkr.ecr.us-west-2.amazonaws.com/huggingface-pytorch-tgi-inference:1.13.1-optimum0.0.20-neuronx-py310-ubuntu22.04` |


Please replace `763104351884` with the correct [AWS account ID](https://github.com/aws/sagemaker-python-sdk/blob/master/src/sagemaker/image_uri_config/huggingface-neuronx.json) and `region` with the AWS region you are working in.
Please replace `763104351884` with the correct [AWS account ID](https://github.com/aws/sagemaker-python-sdk/blob/master/src/sagemaker/image_uri_config/huggingface-neuronx.json) and `region` with the AWS region you are working in.

0 comments on commit fa21917

Please sign in to comment.