Repository navigation
Conversation
out_of_memory() runs the OOM notifier chain with oom_lock held. On a PV domU the balloon's notifier deflates through balloon_deflate() -> xenmem_reservation_va_mapping_update() -> set_phys_to_machine(), and when the page's p2m leaf is still the shared p2m_missing page that allocates the leaf with a GFP_KERNEL __get_free_page(). Reclaim has already failed, so that allocation goes to __alloc_pages_may_oom(), whose mutex_trylock(&oom_lock) fails because we hold it; it counts that as progress, sleeps a jiffy and retries, forever. The OOM killer never runs and the guest hangs instead of killing a process. Pages with a missing leaf sit on the balloon in any PV domU built with maxmem above memory and without the unpopulated allocator, whose extra memory regions are appended at boot, and in one that hotplugged memory it has not populated yet. PVH and HVM guests are not affected, as the va-mapping update is a no-op there, nor is dom0, which does not register the notifier. Deflate under memalloc_noreclaim_save(): with PF_MEMALLOC set the allocator neither reclaims nor enters the OOM path, so the leaf either comes from the reserves or fails, and set_phys_to_machine() then fails the page cleanly. A leaf taken from the reserves lets 512 pages come back, so a memory-freeing context is the right one for this work. Signed-off-by: Ariadne Conill <[email protected]>
…olders gntdev_relocate_map() frees the old placeholder pages once gnttab_unmap_refs() returns 0, but that function returns 0 whenever the hypercall itself succeeded, whether or not every unmap_op in the batch was honoured; a refused op reports its failure in its own status and leaves its grant mapped. A page whose frame still carries a foreign mapping then goes back to the unpopulated pool, and the next use of that gfn has Xen crash the domain. Treat any op with a status other than GNTST_okay, for the kernel unmap ops too where they exist, as the failure it is, so the existing path that leaks the old pages and fails the map handles it. Signed-off-by: Ariadne Conill <[email protected]>
kaniini
requested review from
antoineco,
azenla,
bleggett,
sveith and
tycho
as code owners
November 13, 2026 22:35
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two fixes from the Mythos audit of the Xen balloon work.
OOM notifier livelock on PV domU (ENG-220).
out_of_memory()calls the notifier chain withoom_lockheld. On PV, deflating a page whose p2m leaf is still the shared missing page allocates the leaf withGFP_KERNELfromset_phys_to_machine(). Reclaim has already failed, so the allocation reaches__alloc_pages_may_oom(), whosemutex_trylock(&oom_lock)fails because we hold it; it counts that as progress and retries forever, and the OOM killer never runs. Such pages sit on the balloon in any PV domU built withmaxmemabovememorywithout the unpopulated allocator, or with hotplugged memory not yet populated. The deflate now runs undermemalloc_noreclaim_save(): withPF_MEMALLOCthe allocator neither reclaims nor enters the OOM path, so the leaf comes from the reserves or fails cleanly. A leaf that lets 512 pages come back repays a reserve page at once. PVH, HVM and dom0 are unaffected.gntdev relocation trusts the hypercall return (from the ENG-220 notes).
gntdev_relocate_map()freed the old placeholder pages whenevergnttab_unmap_refs()returned 0, but that only says the batch was processed; a refused op reports in its own status and leaves its grant mapped, and the page would then go back to the unpopulated pool with a live foreign mapping. Each op's status, and the kernel unmap ops' where present, is now checked and a failure takes the existing leak-and-fail path.Built with the latest
linux-kernel-ocix86_64zone.config(XEN_BALLOON,XEN_BALLOON_MEMORY_HOTPLUG,XEN_GNTDEV,XEN_UNPOPULATED_ALLOCall on): full vmlinux, no warnings. Not runtime-tested; reproducing needs a PV domU withmaxmem > memoryandCONFIG_XEN_UNPOPULATED_ALLOC=n, driven to OOM.Not in this change: the block-pool erosion noted on the issue, where every refused dom0 grow breaks up to 32 blocks into single pages that never re-form. That needs a design decision about when blocks are split, and is left open.
ENG-220