Investigate ClusterStorageContainer from kserve

[lampajr]

Investigate/explore the ClusterStorageContainer capability that kserve introduced in the latest release.

Idea behind this exploration:

• Provide a custom storage-initializer that will match URI format like model-registry:// [1]
• This custom image "understands" the URI-specific format and it will fetch model registry data in order to retrieve the correct indexed modelArtifact - the URI should contains required information to retrieve it.
• Download the model using already existing kserve functionalities (the custom image will a wrapper of the existing one that understands the model registry URI format that we can define)
• In this way the InferenceService that will use model registry could be something like (already supported by kserve):

apiVersion: "serving.kserve.io/v1beta1"
kind: "InferenceService"
metadata:
  name: "onnx-model"
spec:
  predictor:
    model:
      modelFormat:
        name: onnx
      storageUri: "model-registry://........"

[1] https://kserve.github.io/website/latest/modelserving/storage/storagecontainers/

[lampajr]

As first iteration I was able to create a custom storage initializer that interacts with the deployed model registry, this was we could create InferenceService using a custom storageUri with a model registry specific format like model-registry://.

Created https://github.com/lampajr/model-registry-storage-initializer that showcase the usage of the initializer with a simple Quickstart based on a plain Kserve installation.

Next step: try to demonstrate the integration with the ODH platform.

[lampajr]

I generalized the implementation of this custom storage initializer to make usage of the model registry REST interface such that any model registry (which is compliant with the openapi spec) is suitable to be adopted.

[lampajr]

Keeping track of this exploration on Kserve repo as well: kserve/kserve#3343

[lampajr]

Additional findings:

• The feature works well on Kserve deployments, this model-registry-storage-initializer works for public available models.
• The feature seems NOT supported in ModelMesh deployments, opened new issue in modelmesh-serving to keep track of this.

As I the feasibility on Kserve side is proven using the created custom storage initializer image I would close this issue as the exploration/experimentation can be considered succeeded for now.

Opened a different issue for the ModelMesh exploration Created issue to further investigate similar enhancement for ModelMesh deployments https://github.com/opendatahub-io/model-registry/issues/262