S3 Destination

Jump to bottom

Alon Gubkin edited this page Mar 1, 2022 · 2 revisions

The S3 Destination allows you to store inference data in S3-compatible storage such as AWS S3, Google Cloud Storage, Azure Blob Storage, or Minio.

apiVersion: inferencedb.aporia.com/v1alpha1
kind: InferenceLogger
metadata:
  name: my-model-inference-logger
  namespace: default
spec:
  topic: my-model
  events:
    type: kserve
    config: {}
  destination:
    type: confluent-s3
    config:
      url: s3://my-bucket/inferencedb
      format: parquet
      awsRegion: us-east-2
      connector: {}

Note the destination object:

Currently, the only supported format is parquet.
If you'd like to customize this even further, you can add any Confluent S3 Sink configuration option to the destination.config.connector dict.

See confluent_s3_destination.py.