Skip to content

Commit

Permalink
Merge pull request #131 from adelton/issue-94
Browse files Browse the repository at this point in the history
The PoC and Pipeline is not about AzureML specifically.
  • Loading branch information
adelton authored Oct 6, 2023
2 parents c5c8082 + dd64158 commit f82cdd7
Show file tree
Hide file tree
Showing 11 changed files with 23 additions and 23 deletions.
4 changes: 2 additions & 2 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -162,6 +162,6 @@ cython_debug/
*-secret.yaml
*-secret.yml
aws-storage-config
pipelines/tekton/azureml-container-pipeline/azureml-container-aws-creds-real.yaml
pipelines/tekton/azureml-container-pipeline/aws-env-real.yaml
pipelines/tekton/build-container-image-pipeline/build-container-image-aws-creds-real.yaml
pipelines/tekton/build-container-image-pipeline/aws-env-real.yaml
oc-debug-pod.yaml
2 changes: 1 addition & 1 deletion acm/odh-edge/base/namespace.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -2,4 +2,4 @@ apiVersion: v1
kind: Namespace
metadata:
# edit this to the namespace where the model is deployed
name: azureml-model-to-edge
name: model-to-edge
22 changes: 11 additions & 11 deletions pipelines/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ They are also referenced by these names in the example pipeline YAML files.
## Prerequisites

- OpenShift cluster with [OpenShift Pipelines Operator](https://docs.openshift.com/container-platform/4.13/cicd/pipelines/installing-pipelines.html) installed
- OpenShift project / namespace. E.g. `oc new-project azureml-model-to-edge`
- OpenShift project / namespace. E.g. `oc new-project model-to-edge`
- A repository on [Quay.io](https://quay.io/)
- S3 bucket for storing the models
- A clone of this repository
Expand All @@ -41,32 +41,32 @@ Create an S3 bucket and upload the directories with the models:

![S3 models example](../.github/images/S3-models.png)

Fill information about access to your S3 bucket in a copy of [`aws-env.yaml`](tekton/azureml-container-pipeline/aws-env.yaml).
Fill information about access to your S3 bucket in a copy of [`aws-env.yaml`](tekton/build-container-image-pipeline/aws-env.yaml).
If you don't have or know your access key, generate one in AWS account's Security credentials > Access keys.

Then store the credentials in an OpenShift secret:

```bash
cp tekton/azureml-container-pipeline/aws-env.yaml tekton/azureml-container-pipeline/aws-env-real.yaml
vi tekton/azureml-container-pipeline/aws-env-real.yaml
oc create -f tekton/azureml-container-pipeline/aws-env-real.yaml
cp tekton/build-container-image-pipeline/aws-env.yaml tekton/build-container-image-pipeline/aws-env-real.yaml
vi tekton/build-container-image-pipeline/aws-env-real.yaml
oc create -f tekton/build-container-image-pipeline/aws-env-real.yaml
```

### Deploy and run the build pipeline

Update the `aws-bucket-name` parameter value from its default `rhoai-edge-models` in
[`azureml-container-pipelinerun-bike-rentals.yaml`](tekton/azureml-container-pipeline/azureml-container-pipelinerun-bike-rentals.yaml)
[`build-container-image-pipelinerun-bike-rentals.yaml`](tekton/build-container-image-pipeline/build-container-image-pipelinerun-bike-rentals.yaml)
and/or
[`azureml-container-pipelinerun-tensorflow-housing.yaml`](tekton/azureml-container-pipeline/azureml-container-pipelinerun-tensorflow-housing.yaml)
[`build-container-image-pipelinerun-tensorflow-housing.yaml`](tekton/build-container-image-pipeline/build-container-image-pipelinerun-tensorflow-housing.yaml)
to match your S3 bucket name.

Then create the pipeline(s) to build the container image with AI runtime:

```bash
oc apply -k tekton/azureml-container-pipeline/
oc create -f tekton/azureml-container-pipeline/azureml-container-pipelinerun-bike-rentals.yaml
oc apply -k tekton/build-container-image-pipeline/
oc create -f tekton/build-container-image-pipeline/build-container-image-pipelinerun-bike-rentals.yaml
# and / or
oc create -f tekton/azureml-container-pipeline/azureml-container-pipelinerun-tensorflow-housing.yaml
oc create -f tekton/build-container-image-pipeline/build-container-image-pipelinerun-tensorflow-housing.yaml
```

Check what objects were created and what pipelines executed either in OpenShift Console
Expand All @@ -82,7 +82,7 @@ oc get imagestream
```
and then run `oc describe` on them, for example
```
oc describe pipelinerun.tekton.dev/azureml-container-bike-rentals-66q8n
oc describe pipelinerun.tekton.dev/build-container-image-bike-rentals-66q8n
# or
oc describe imagestream/tensorflow-housing
```
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
apiVersion: tekton.dev/v1beta1
kind: Pipeline
metadata:
name: azureml-container
name: build-container-image
spec:
params:
- name: model-name
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,8 @@ apiVersion: tekton.dev/v1beta1
kind: PipelineRun
metadata:
labels:
tekton.dev/pipeline: azureml-container
generateName: azureml-container-bike-rentals-
tekton.dev/pipeline: build-container-image
generateName: build-container-image-bike-rentals-
spec:
params:
- name: model-name
Expand All @@ -21,7 +21,7 @@ spec:
- name: containerfileRelativePath
value: pipelines/containerfiles/Containerfile.seldonio.mlserver.mlflow
pipelineRef:
name: azureml-container
name: build-container-image
serviceAccountName: pipeline
timeout: 1h0m0s
workspaces:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,8 @@ apiVersion: tekton.dev/v1beta1
kind: PipelineRun
metadata:
labels:
tekton.dev/pipeline: azureml-container
generateName: azureml-container-tensorflow-housing-
tekton.dev/pipeline: build-container-image
generateName: build-container-image-tensorflow-housing-
spec:
params:
- name: model-name
Expand All @@ -21,7 +21,7 @@ spec:
- name: containerfileRelativePath
value: pipelines/containerfiles/Containerfile.openvino.mlserver.mlflow
pipelineRef:
name: azureml-container
name: build-container-image
serviceAccountName: pipeline
timeout: 1h0m0s
workspaces:
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
kind: Kustomization
apiVersion: kustomize.config.k8s.io/v1beta1
resources:
- azureml-container-buildah-cache-pvc.yaml
- build-container-image-buildah-cache-pvc.yaml
- kserve-download-model.yaml
- check-model-and-containerfile-exists.yaml
- azureml-container-pipeline.yaml
- build-container-image-pipeline.yaml

0 comments on commit f82cdd7

Please sign in to comment.