Hello! This repository shows how to deploy sample CDK app that has live SageMaker endpoint with LLM (by default, Llama2-7b chat).
- Install CDK
pip install -r requirements.txt
pip install -r requirements-dev.txt
To deploy, run:
cdk deploy
If asked Do you wish to deploy these changes
enter y
To invoke endpoint directly, run:
python3 invoke_sagemaker.py
The script will automatically read endpoint name from output of CDK
Run
cdk destroy
And when asked for approval, enter y