Multi Lora Adapter Hosting with Sagemaker

This repo provides example notebooks how to deploy multiple Lora adapters with Sagemaker Studio.

Getting Started - Notebooks

• Deploy single model with optimized inference and DJL Serving – Only HF_MODEL_ID required
• Deploying 100's of Lora adapters with Lorax Server and Sagemaker
• Multi Adapter Deployment with DJL Serving

Blogs

• Efficient and cost-effective multi-tenant LoRA serving with Amazon SageMaker
• Overivew + LoRA Serving on Amazon SageMaker — Serve 100’s of Fine-Tuned LLMs For the Price of 1
• LoRA Exchange (LoRAX): Serve 100s of Fine-Tuned LLMs for the Cost of 1

Papers

• Punica: Multi-Tenant LoRA Serving
• S-LoRA: Serving Thousands of Concurrent LoRA Adapters

Other Repos

• Serving LoRA-based Llama 2 and Mistral adapters with high performance on SageMaker
• LoRAX Server