Mypy and pip-installed type stubs

[calliecameron]

I'm trying to write a test that runs mypy. I thought I had the following working in rules_python 0.22.0:

$ ls -A
.bazelversion  BUILD  MODULE.bazel  WORKSPACE  bar.py  foo.py  lint.bzl  requirements.txt  stub.sh

$ cat .bazelversion
6.2.1

$ cat BUILD
load("@rules_python//python:pip.bzl", "compile_pip_requirements")
load("@pip//:requirements.bzl", "entry_point", "requirement")
load("@rules_python//python:defs.bzl", "py_library")
load("lint.bzl", "py_library_with_lint")

compile_pip_requirements(
    name = "requirements",
    requirements_in = "requirements.txt",
    requirements_txt = "requirements_lock.txt",
    tags = ["requires-network"],
)

alias(
    name = "mypy",
    actual = entry_point("mypy"),
)

py_library_with_lint(
    name = "foo",
    srcs = ["foo.py"],
    deps = [
         ":bar",
         requirement("tabulate"),
    ],
)

py_library(
    name = "bar",
    srcs = ["bar.py"],
)

$ cat MODULE.bazel
module(
    name = "example",
    version = "0.0.0",
)

bazel_dep(
    name = "rules_python",
    version = "0.22.0",
)

pip = use_extension("@rules_python//python/extensions:pip.bzl", "pip")
pip.parse(
    name = "pip",
    requirements_lock = "//:requirements_lock.txt",
)
use_repo(pip, "pip")

$ cat WORKSPACE

$ cat bar.py
def bar(s: str) -> str:
    return s + "!"

$ cat foo.py
import sys
import tabulate
import bar


def foo() -> str:
    return bar.bar(tabulate.tabulate([["foo"]]))

$ cat lint.bzl
load("@rules_python//python:defs.bzl", "py_library")

def py_library_with_lint(name, **kwargs):
    py_library(
        name = name,
        **kwargs
    )

    srcs = kwargs.get("srcs", [])
    deps = kwargs.get("deps", [])

    native.sh_test(
        name = name + "_mypy_test",
        srcs = ["stub.sh"],
        args = [
            "$(rootpath :mypy)",
            "--strict",
            "--explicit-package-bases",
            "--scripts-are-modules",
        ] + ["$(location %s)" % src for src in srcs],
        data = [
            ":mypy",
        ] + srcs + deps,
    )

$ cat requirements.txt
mypy == 1.4.1
tabulate == 0.9.0
types-tabulate == 0.9.0.2

$ cat stub.sh
#!/bin/bash

"${@}"

I would then do the following:

$ touch requirements_lock.txt
$ bazel run --enable_bzlmod :requirements.update
$ bazel test --enable_bzlmod :all

...and the mypy test would pass. Now it turns out this only ever worked because I was running bazel test from inside a virtualenv where I'd run pip install -r requirements.txt. I.e. because I was using the system python, not a hermetic toolchain, types-tabulate installed in the virtualenv was leaking into the test environment, and the test was passing. If I deactivate the virtualenv, the test fails - and adding requirement("types-tabulate") to the library doesn't help.

Switching to rules_python 0.24.0, with the following diff:

$ diff MODULE.bazel.old MODULE.bazel
8c8
<     version = "0.22.0",
---
>     version = "0.24.0",
13c13
<     name = "pip",
---
>     hub_name = "pip",
16c16
< use_repo(pip, "pip")
---
> use_repo(pip, "pip", "pip_311")
$ diff BUILD.old BUILD
2c2,3
< load("@pip//:requirements.bzl", "entry_point", "requirement")
---
> load("@pip//:requirements.bzl", "requirement")
> load("@pip_311//:requirements.bzl", "entry_point")

...and running:

$ bazel run --enable_bzlmod :requirements.update
$ bazel test --enable_bzlmod :all

...the test fails, regardless of whether I'm in the virtualenv or not, with:

exec ${PAGER:-/usr/bin/less} "$0" || exit 1
Executing tests from //:foo_mypy_test
-----------------------------------------------------------------------------
foo.py:2: error: Library stubs not installed for "tabulate"  [import]
foo.py:2: note: Hint: "python3 -m pip install types-tabulate"
foo.py:2: note: (or run "mypy --install-types" to install all missing stub packages)
foo.py:2: note: See https://mypy.readthedocs.io/en/stable/running_mypy.html#missing-imports
Found 1 error in 1 file (checked 1 source file)

...because the new version of rules_python uses a toolchain by default, and isn't leaking types-tabulate from the virtualenv into the test environment. Note that the standard library import and local import work fine, only the pip-installed stubs are missing.

I'd like to use a toolchain anyway, so I can pin a specific python version, so now my question is: how do I do this properly? Is using a stub shell script the right way to call an entry_point in a test, and how do I get mypy to find the pip-installed stubs?

I'm doing a similar thing with a pylint test, and it's failing with errors like [E0401(import-error), ] Unable to import 'tabulate'. On the other hand, linters that don't follow imports (e.g. black) work fine.

OS: Ubuntu 22.04
Bazel version: 6.2.1
rules_python version: 0.24.0

[alexeagle]

FYI I'm adding mypy to rules_lint: https://github.com/aspect-build/rules_lint/issue/79

[calliecameron]

Thank, I'll keep an eye on that. Working link: aspect-build/rules_lint#79.

In the meantime I found a workaround - running mypy from a stub python script, rather than using the entry_point. py_test sets up the environment so that mypy can find everything, including the type stubs if specified.

$ ls -A
.bazelversion  BUILD  MODULE.bazel  WORKSPACE  bar.py  foo.py  lint.bzl  mypy_stub.py  requirements.txt

$ cat .bazelversion 
6.2.1

$ cat BUILD
load("@rules_python//python:pip.bzl", "compile_pip_requirements")
load("@pip//:requirements.bzl", "requirement")
load("@rules_python//python:defs.bzl", "py_library")
load("lint.bzl", "py_library_with_lint")

compile_pip_requirements(
    name = "requirements",
    requirements_in = "requirements.txt",
    requirements_txt = "requirements_lock.txt",
    tags = ["requires-network"],
)

py_library_with_lint(
    name = "foo",
    srcs = ["foo.py"],
    deps = [
         ":bar",
         requirement("tabulate"),
    ],
    type_stub_deps = [requirement("types-tabulate")],
)

py_library(
    name = "bar",
    srcs = ["bar.py"],
)

$ cat MODULE.bazel
module(
    name = "example",
    version = "0.0.0",
)

bazel_dep(
    name = "rules_python",
    version = "0.24.0",
)

python = use_extension("@rules_python//python/extensions:python.bzl", "python")
python.toolchain(
    python_version = "3.10",
)

pip = use_extension("@rules_python//python/extensions:pip.bzl", "pip")
pip.parse(
    hub_name = "pip",
    python_version = "3.10",
    requirements_lock = "//:requirements_lock.txt",
)
use_repo(pip, "pip", "pip_310")

$ cat WORKSPACE 

$ cat bar.py 
def bar(s: str) -> str:
    return s + "!"

$ cat foo.py 
import sys
import tabulate
import bar


def foo() -> str:
    return bar.bar(tabulate.tabulate([["foo"]]))

$ cat lint.bzl 
load("@rules_python//python:defs.bzl", "py_library", "py_test")
load("@pip//:requirements.bzl", "requirement")

def py_library_with_lint(name, type_stub_deps=None, **kwargs):
    py_library(
        name = name,
        **kwargs
    )

    srcs = kwargs.get("srcs", [])
    deps = kwargs.get("deps", [])
    type_stub_deps = type_stub_deps or []

    py_test(
        name = name + "_mypy_test",
        srcs = ["mypy_stub.py"],
        main = "//:mypy_stub.py",
        deps = deps + type_stub_deps + [requirement("mypy")],
        data = srcs,
        args = [
            "--strict",
            "--explicit-package-bases",
            "--scripts-are-modules",
        ] + ["$(location %s)" % src for src in srcs],
    )

$ cat mypy_stub.py 
# from https://mypy.readthedocs.io/en/stable/extending_mypy.html
import sys
from mypy import api

result = api.run(sys.argv[1:])

if result[0]:
    print('\nType checking report:\n')
    print(result[0])  # stdout

if result[1]:
    print('\nError report:\n')
    print(result[1])  # stderr

print('\nExit status:', result[2])

sys.exit(result[2])

$ cat requirements.txt 
mypy == 1.4.1
tabulate == 0.9.0
types-tabulate == 0.9.0.2

Then as before:

$ touch requirements_lock.txt
$ bazel run --enable_bzlmod :requirements.update
$ bazel test --enable_bzlmod :all

and the test passes.

[github-actions]

This issue has been automatically marked as stale because it has not had any activity for 180 days. It will be closed if no further activity occurs in 30 days.
Collaborators can add an assignee to keep this open indefinitely. Thanks for your contributions to rules_python!

[aignas]

I'll keep this open for pyi file inclusion in py_library. I am not sure if there is a good story yet, so it may be useful to consider these in rules_python.