Updated conformance tests and results for latest versions of type che… #1863
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…ckers. One test case in the `generics_syntax_infer_variance` was commented out temporarily until we decide how to handle variance inference for dataclasses in Python 3.13. Mypy version was updated from 1.11.1 to 1.11.2. Minor error message change but no other substantive changes to the conformance results. Pyright version was updated from 1.1.374 to 1.1.384. Many error messages changed slightly. Tests also identified three conformance regressions. Pytype version was updated from 2024.04.11 to 2024.09.13. Error message format changed significantly, requiring updates to the test logic. No other substantive changes to conformance results. Changed test infrastructure to make sure old versions of type checkers are uninstalled before installing new to guarantee correct version and increase determinism of timings.
There was a problem hiding this comment.
On macOS there's an issue where a fresh install of (mypyc-compiiled) mypy will trigger a malware scanner and so the first time will often be really slow. In mypy_primer I work around this by running mypy --version once before recording any times.
We should maybe strip ANSI codes from the pytype output
JelleZijlstra
approved these changes
Oct 10, 2024
| Line 68: Unexpected errors ['File "aliases_implicit.py", line 68, in good_type_aliases: Callable[Concatenate, Any] [assert-type]'] | ||
| Line 72: Unexpected errors ['File "aliases_implicit.py", line 72, in good_type_aliases: Callable[[Any], None] [assert-type]'] | ||
| Line 105: Unexpected errors ['File "aliases_implicit.py", line 105, in <module>: Invalid type annotation \\'[int, str]\\' for p2 [invalid-annotation]', 'File "aliases_implicit.py", line 105, in <module>: Invalid type annotation \\'True\\' for p10 [invalid-annotation]', 'File "aliases_implicit.py", line 105, in <module>: Invalid type annotation \\'<instance of List[Type[int]]>\\' for p4 [invalid-annotation]', 'File "aliases_implicit.py", line 105, in <module>: Invalid type annotation \\'3\\' for p9 [invalid-annotation]', 'File "aliases_implicit.py", line 105, in <module>: Invalid type annotation \\'1\\' for p11 [invalid-annotation]', 'File "aliases_implicit.py", line 105, in <module>: Invalid type annotation \\'((int, str),)\\' for p3 [invalid-annotation]', 'File "aliases_implicit.py", line 105, in <module>: Invalid type annotation "{\\'a\\': \\'b\\'}" for p5 [invalid-annotation]'] | ||
| Line 54: Unexpected errors ["aliases_implicit.py:54:9: \\x1b[1m\\x1b[31merror\\x1b[39m\\x1b[0m: in <module>: Invalid type annotation '[str, str]' [invalid-annotation]", "aliases_implicit.py:54:9: \\x1b[1m\\x1b[31merror\\x1b[39m\\x1b[0m: in <module>: Invalid type annotation 'Callable[Concatenate[int, P], R][[str, str], None]' [invalid-annotation]"] |
There was a problem hiding this comment.
Does this coloring show up correctly in the output tables we generate? It might be better to configure pytype to not use color, if possible.
There was a problem hiding this comment.
No, this doesn't show up in the output report.
Unfortunately, I don't see a way to disable color. If there is, it's not documented.
…the artificial time associated with malware scanning on MacOS.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
…ckers.
One test case in the
generics_syntax_infer_variancewas commented out temporarily until we decide how to handle variance inference for dataclasses in Python 3.13.Mypy version was updated from 1.11.1 to 1.11.2. Minor error message change but no other substantive changes to the conformance results.
Pyright version was updated from 1.1.374 to 1.1.384. Many error messages changed slightly. Tests also identified three conformance regressions.
Pytype version was updated from 2024.04.11 to 2024.09.13. Error message format changed significantly, requiring updates to the test logic. No other substantive changes to conformance results.
Changed test infrastructure to make sure old versions of type checkers are uninstalled before installing new to guarantee correct version and increase determinism of timings.