gh-151728: Clear the typing caches at interpreter shutdown - #155002
Open
wjakob wants to merge 1 commit into
Open
gh-151728: Clear the typing caches at interpreter shutdown#155002wjakob wants to merge 1 commit into
wjakob wants to merge 1 commit into
Conversation
Re-apply pythonGH-154858, which was reverted in pythonGH-154992 because it broke the reference leak buildbots. ``atexit.register(_clear_caches)`` runs on every import of ``typing``, and the registered handler reaches the module dict through ``_clear_caches.__globals__``. Any throwaway copy of ``typing`` therefore stays alive until interpreter shutdown. Two tests create such a copy on each iteration: - ``InternalsTests.test_collect_parameters`` imports a fresh ``typing``. - ``CollectionsAbcTests.test_bytestring`` drops ``typing`` from ``sys.modules`` and re-imports it. Both now unregister the exit handler of the copy they created.
This was referenced Jul 31, 2026
Merged
JelleZijlstra
added
the
🔨 test-with-refleak-buildbots
|
🤖 New build scheduled with the buildbot fleet by @JelleZijlstra for commit f42f407 🤖 Results will be shown at: https://buildbot.python.org/all/#/grid?branch=refs%2Fpull%2F155002%2Fmerge If you want to schedule another build, you need to add the 🔨 test-with-refleak-buildbots label again. |
bedevere-bot
removed
the
🔨 test-with-refleak-buildbots
Member
|
Makes sense, I triggered the refleak buildbots to confirm |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
typingcaches every subscripted type in module-levellru_cachees. Those caches are only released when thetypingmodule itself is torn down, which does not happen when some extension module leaks a reference to it. Sincetypingis nearly universal, a single such leak (e.g. viatorch) keeps the cached types alive past interpreter shutdown, including types owned by unrelated, correctly written extensions. Those extensions then look like they are leaking:valgrindreports them, and nanobind prints warnings about types, functions, and instances that were never freed.typing._clear_caches()already exists but nothing calls it at shutdown. This PR registers it withatexitso that the caches are dropped during finalization and the process terminates leak-free. This complements gh-98253, which broke a cycle inside_tp_cachebut left the retention itself in place.This re-lands GH-154858, which was reverted in GH-154992 because of reference leak failures. The problem there was that the handler runs on every import of
typingand keeps the module alive through its__globals__. Two tests intest_typingimport a throwaway copy oftyping, which then leaks once per repetition under-R. Both now unregister the handler of the copy they made.The PR adds no test because reproducing the problem requires an extension module that deliberately leaks a reference (i.e. one with a missing
tp_traverse). A minimal one is at https://github.com/wjakob/typing-leak.typingcaches keep extension types alive past shutdown #151728