Start adding JSON Schema configuration

[bollwyvl]

References

• start Generate JSON Schema #3 in earnest

Changes

• packaging
  • adds [schema] extra, replicated in [dev]
• spec
  • uses Field for just about everything
  • adds schema_extra to some top-level objects
    • the top lock spec itself gets
      • $id: just randomly picked one
      • $schema: (chose draft 19 because it supports deprecated, but is in all version of jsonschema >=4)
    • all base models
      • some extra required fields (fields with defaults don't get required, according to pydantic)
      • description
      • examples (if relevant)

        • not strictly in the schema spec, but are often hoisted to UI, docs, tests, etc.

• test
  • validate historical specs against schema
  • add conftest.py with some fixtures
  • add failing test of check_wheel_filenames to get up to 100% coverage

    • both historic specs have name/version issues

Open Questions

• What should the $id be?
• while this is a URI, and not necessarily resolveable, it's nice if it does

Future Work

Things that could happen on this PR, or in follow-ons:

• check in schema

  • does not yet check in the schema, but could. the example is started below

  • there is probably a "canonical" way to sort values that is clearer and would give better diff

• handle a "loose" and "strict" schema

  • e.g. coerced with default values, would require some potentially complex multiple inheritance

• hoist to docs

  • with e.g. sphinx-jsonschema

Strawman Schema

This was generated with:

python -c '\
from pyodide_lock import PyodideLockSpec; \
import json; \
print(json.dumps(PyodideLockSpec.schema(), indent=2, sort_keys=True))
' > schema.json

{
  "$id": "https://pyodide.org/schema/pyodide-lock/v0-lockfile.schema.json",
  "$schema": "https://json-schema.org/draft/2019-09/schema#",
  "additionalProperties": false,
  "definitions": {
    "InfoSpec": {
      "additionalProperties": false,
      "description": "the execution environment in which the packages in this lockfile can be installed",
      "properties": {
        "arch": {
          "default": "wasm32",
          "description": "the short name for the compiled architecture, available in dependency markers as `platform_machine`",
          "enum": [
            "wasm32",
            "wasm64"
          ],
          "title": "Arch",
          "type": "string"
        },
        "platform": {
          "description": "the emscripten virtual machine for which this distribution is  compiled, not available directly in a dependency marker: use e.g. `plaform_system == \"Emscripten\" and platform_release == \"3.1.45\"`",
          "examples": [
            "emscripten_3_1_32",
            "emscripten_3_1_45"
          ],
          "title": "Platform",
          "type": "string"
        },
        "python": {
          "description": "the version of python for which this lockfile is valid, available in version markers as `platform_machine`",
          "examples": [
            "3.11.2",
            "3.11.3"
          ],
          "title": "Python",
          "type": "string"
        },
        "version": {
          "description": "the PEP 440 version of pyodide",
          "examples": [
            "0.24.1",
            "0.23.3"
          ],
          "title": "Version",
          "type": "string"
        }
      },
      "required": [
        "arch",
        "platform",
        "python",
        "version"
      ],
      "title": "InfoSpec",
      "type": "object"
    },
    "PackageSpec": {
      "additionalProperties": false,
      "description": "a single pyodide-compatible file",
      "properties": {
        "depends": {
          "default": [],
          "description": "package names that must be installed when this package in installed",
          "items": {
            "type": "string"
          },
          "title": "Depends",
          "type": "array",
          "uniqueItems": true
        },
        "file_name": {
          "description": "the URL of the file",
          "examples": [
            "pyodide_lock-0.1.0-py3-none-any.whl",
            "https://files.pythonhosted.org/packages/py3/m/micropip/micropip-0.5.0-py3-none-any.whl"
          ],
          "format": "uri-reference",
          "title": "File Name",
          "type": "string"
        },
        "imports": {
          "default": [],
          "description": "the importable names provided by this package.note that PEP 420 namespace packages will likely not be correctly found.",
          "items": {
            "type": "string"
          },
          "title": "Imports",
          "type": "array"
        },
        "install_dir": {
          "default": "site",
          "description": "the file system destination for a package's data",
          "examples": [
            "dynlib",
            "stdlib"
          ],
          "title": "Install Dir",
          "type": "string"
        },
        "name": {
          "description": "the verbatim name as found in the package's metadata",
          "examples": [
            "pyodide-lock",
            "PyYAML",
            "ruamel.yaml"
          ],
          "title": "Name",
          "type": "string"
        },
        "package_type": {
          "default": "package",
          "description": "the top-level kind of content provided by this package",
          "enum": [
            "package",
            "cpython_module",
            "shared_library",
            "static_library"
          ],
          "title": "Package Type",
          "type": "string"
        },
        "sha256": {
          "description": "the SHA256 cryptographic hash of the file",
          "title": "Sha256",
          "type": "string"
        },
        "shared_library": {
          "default": false,
          "deprecated": true,
          "description": "(deprecated) whether this package is a shared library. replaced with `package_type: shared_library`",
          "title": "Shared Library",
          "type": "boolean"
        },
        "unvendored_tests": {
          "default": false,
          "description": "whether the package's tests folder have been repackaged as a separate archive",
          "title": "Unvendored Tests",
          "type": "boolean"
        },
        "version": {
          "description": "the reported version of the package",
          "examples": [
            "0.1.0",
            "1.0.0a0",
            "1.0.0a0.post1"
          ],
          "title": "Version",
          "type": "string"
        }
      },
      "required": [
        "depends",
        "file_name",
        "imports",
        "install_dir",
        "name",
        "sha256",
        "version"
      ],
      "title": "PackageSpec",
      "type": "object"
    }
  },
  "description": "a description of a viable pyodide runtime environment, as defined by pyodide-lock",
  "properties": {
    "info": {
      "allOf": [
        {
          "$ref": "#/definitions/InfoSpec"
        }
      ],
      "description": "the execution environment in which the packages in this lockfile can be installable",
      "title": "Info"
    },
    "packages": {
      "additionalProperties": {
        "$ref": "#/definitions/PackageSpec"
      },
      "default": {},
      "description": "a set of packages keyed by name",
      "title": "Packages",
      "type": "object"
    }
  },
  "required": [
    "info"
  ],
  "title": "PyodideLockSpec",
  "type": "object"
}