Repository navigation
[mypyc] Report errors for native classes with unsupported bases - #22374
Merged
Merged
Conversation
…ases Fixes mypyc/mypyc#1186. Fixes mypyc/mypyc#1103. A native class that inherits from `collections.defaultdict` was treated like a regular `dict` subclass, but `defaultdict` instances have an extra field, so the native instance layout was too small and compiled code could segfault or behave incorrectly: ```python from collections import defaultdict class DD(defaultdict[str, int]): def __init__(self) -> None: super().__init__(int) DD()["x"] # Segfault ``` Inheriting from `collections.OrderedDict` has the same problem (the compiled module failed to import). The problem is more general: native classes assume that non-mypyc base classes have the layout of `object` (or `dict`/`BaseException`), which isn't true for many classes implemented in C (e.g. `random.Random`, `io.BytesIO`, `datetime.date`). Stubs don't describe instance layouts, so now a native class can only inherit from a non-mypyc class defined in a stub if the class is in an allowlist (`SAFE_STUB_BASE_CLASSES` in `mypyc/irbuild/safe_bases.py`). Otherwise mypyc reports an error with the usual note about using `@mypyc_attr(native_class=False)` as a workaround. This also covers `defaultdict` and `OrderedDict`. Classes defined in Python sources are still accepted, unless they define `__slots__` or are compiled separately into a C extension. The allowlist only includes standard library classes with evidence of being subclassed in real code (at least two of the 1000 most downloaded PyPI projects, a large private codebase, or mypy itself) that are documented in the CPython docs and have a compatible runtime layout on Python 3.10-3.15, including the free-threaded build. Native subclasses of each entry were compiled and exercised on Python 3.10, 3.14 (including free-threaded) and 3.15. This keeps the list size manageable and avoids internal classes that are likely to change. The entries are the full names mypy uses (those of the modules where typeshed defines the classes), so a test checks that they still match how mypy resolves the classes on the Python version and platform the tests run on. Classes that keep weak references to the instance (`threading.Thread`, logging handlers, ...) are left out for now and listed in a comment, since weakrefs to native subclasses of interpreted classes aren't supported yet.
p-sawicki
approved these changes
Nov 5, 2026
Comment on lines
+628
to
+629
| if is_compiled_module_path(path): | ||
| return base, "defined in a separately compiled extension module" |
Collaborator
There was a problem hiding this comment.
should this be something like "defined in an extension module not produced by mypyc"? "separately compiled" could be confusing considering mypyc has a separate compilation mode.
Contributor
|
According to mypy_primer, this change doesn't affect type check results on a corpus of open source code. ✅ |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes mypyc/mypyc#1186. Fixes mypyc/mypyc#1103.
A native class that inherits from
collections.defaultdictwas treated like a regulardictsubclass, butdefaultdictinstances have an extra field, so the native instance layout was too small and compiled code could segfault or behave incorrectly:Inheriting from
collections.OrderedDicthas the same problem (the compiled module failed to import).The problem is more general: native classes assume that non-mypyc base classes have the layout of
object(ordict/BaseException), which isn't true for many classes implemented in C (e.g.random.Randomorio.BytesIO). Stubs don't describe instance layouts, so now a native class can only inherit from a non-mypyc class defined in a stub if the class is in an allowlist. Otherwise mypyc reports an error with the usual note about using@mypyc_attr(native_class=False)as a workaround. This also coversdefaultdictandOrderedDict. Classes defined in Python sources are still accepted, unless they define__slots__or are compiled separately into a C extension.The allowlist only includes standard library classes with evidence of being subclassed in real code (at least two of the 1000 most downloaded PyPI projects, a large private codebase, or mypy itself) that are documented in the CPython docs and have a compatible runtime layout on Python 3.10-3.15, including the free-threaded build. Native subclasses of each entry were compiled and exercised on Python 3.10, 3.14 (including free-threaded) and 3.15. This keeps the list size manageable and avoids internal classes that are likely to change.
The entries are the full names mypy uses (those of the modules where typeshed defines the classes), so a test checks that they still match how mypy resolves the classes on the Python version and platform the tests run on.
Classes that keep weak references to the instance (
threading.Thread, logging handlers, ...) are left out for now and listed in a comment, since weakrefs to native subclasses of interpreted classes aren't supported yet.