Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix deserialization of anyOf types with lists #30

Merged
merged 1 commit into from
Nov 1, 2023

Conversation

davidmigloz
Copy link
Contributor

@davidmigloz davidmigloz commented Nov 1, 2023

It was good that you added support for deserializing anyOf types because there's also a case for that in the OpenAI API (although they have not adjusted the spec appropriately yet - openai/openai-openapi#99).

When you create an embedding, OpenAI can return the embedding as a list of floats or as a base64-encoded string. So the schema looks like this:

Embedding:
  type: object
  description: |
    Represents an embedding vector returned by embedding endpoint.
  properties:
    index:
      type: integer
      description: The index of the embedding in the list of embeddings.
    embedding:
      title: EmbeddingVector
      description: |
        The embedding vector, which is a list of floats. The length of vector depends on the model as listed in the [embedding guide](https://platform.openai.com/docs/guides/embeddings).
      oneOf:
        - type: string
          description: The embedding vector as a base64-encoded string.
        - type: array
          description: The embedding vector as a list of floats.
          items:
            type: number
    object:
      type: string
      description: The object type, which is always "embedding".
  required:
    - index
    - object
    - embedding

The previous implementation was generating the following JSON converter:

class _EmbeddingVectorConverter
    implements JsonConverter<EmbeddingVector, Object?> {
  const _EmbeddingVectorConverter();

  @override
  EmbeddingVector fromJson(Object? data) {
    if (data is String) {
      return EmbeddingVector.string(data);
    }
    if (data is List<double>) {
      return EmbeddingVector.arrayNumber(data);
    }
    throw Exception('Unexpected value for EmbeddingVector: $data');
  }

  @override
  Object? toJson(EmbeddingVector data) {
    return switch (data) {
      _UnionEmbeddingVectorString(value: final v) => v,
      _UnionEmbeddingVectorArrayNumber(value: final v) => v,
    };
  }
}

However, it failed to deserialize a list of floats because data is of type List<dynamic>, which is not a List<double>.

This PR changes the condition to check that data is a List and all the items of the list are of the specified type.

This is how the new JSON converter looks like:

class _EmbeddingVectorConverter
    implements JsonConverter<EmbeddingVector, Object?> {
  const _EmbeddingVectorConverter();

  @override
  EmbeddingVector fromJson(Object? data) {
    if (data is String) {
      return EmbeddingVector.string(data);
    }
    if (data is List && data.every((item) => item is double)) {
      return EmbeddingVector.arrayNumber(data.cast());
    }
    throw Exception('Unexpected value for EmbeddingVector: $data');
  }

  @override
  Object? toJson(EmbeddingVector data) {
    return switch (data) {
      _UnionEmbeddingVectorString(value: final v) => v,
      _UnionEmbeddingVectorArrayNumber(value: final v) => v,
    };
  }
}

cc @walsha2

@walsha2 walsha2 merged commit c65459b into tazatechnology:main Nov 1, 2023
1 check passed
@walsha2
Copy link
Contributor

walsha2 commented Nov 1, 2023

openapi_spec (v0.7.2) - Published.

@davidmigloz good find!

@davidmigloz davidmigloz deleted the any-of-deser branch November 2, 2023 09:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants