Skip to content

Latest commit

 

History

History
92 lines (64 loc) · 2.32 KB

README.md

File metadata and controls

92 lines (64 loc) · 2.32 KB
Raw

HuggingFace Diffusion Models on Seldon Core

Requirements

  • Helm (helm)
  • (for this README's example only) Azure CLI ( az ) setup with your login credentials
    • You can use gcloud or aws or even a local k8s cluster if you prefer

Setup

Cluster Setup (Azure Kubernetes Service example)

  1. Setup the resource group and cluster name

    export RESOURCE_GROUP_NAME=<your resource group name of choice>
export CLUSTER_NAME=<your cluster name of choice>

  2. Create a new resource group and a new cluster

    az group create --name $RESOURCE_GROUP_NAME \
    --location eastus
az aks create -g $RESOURCE_GROUP_NAME \
    -n $CLUSTER_NAME --enable-managed-identity \
    --node-count 1 \
    --enable-addons monitoring \
    --generate-ssh-keys

  3. Get the cluster credentials

    az aks get-credentials --resource-group $RESOURCE_GROUP_NAME \
    --name $CLUSTER_NAME

Local Model Setup

  1. Create a folder named artifacts

    mkdir artifacts

  2. Rename your Stable Diffusion to model.safetensors and move it to artifacts

Seldon Core Setup

  1. Build and push the model image to your container registry

    CONTAINER_REGISTRY_URL=<your container registry URL>
CONTAINER_URL=$CONTAINER_REGISTRY_URL/diffusion:v1.0

docker build . -t $CONTAINER_URL
docker push $CONTAINER_URL

    NOTE: If you get a cannot connect to the Docker Daemon error, you may need to specify your Docker socket to s2i using the -U . ex. -U unix:///Users/computer/.docker/run/docker.sock

  2. Install Seldon Core on your cluster

    kubectl create namespace seldon-system

helm install seldon-core seldon-core-operator \
    --repo https://storage.googleapis.com/seldon-charts \
    --set usageMetrics.enabled=true \
    --namespace seldon-system

  3. Create the model namespace

    kubectl create namespace diffusion

  4. Update the <repository URL> in charts/model.yaml

    containers:
    - name: diffusion
      image: <repository URL>/diffusion:v0.1

  5. Apply the charts

    kubectl apply -f charts/model.yaml