Skip to content

Fix memory planning corruption when multiple HOPs have overlapping output lifetimes - #21494

Draft
gtyukasz wants to merge 1 commit into
pytorch:mainfrom
gtyukasz:bugfix/higher-order-op-output-memplan
Draft

Fix memory planning corruption when multiple HOPs have overlapping output lifetimes#21494
gtyukasz wants to merge 1 commit into
pytorch:mainfrom
gtyukasz:bugfix/higher-order-op-output-memplan

Conversation

@gtyukasz

Copy link
Copy Markdown

Fixes #21490

Summary

  • Fix memory corruption when sequential torch.cond nodes have overlapping output lifetimes — the second cond's submodule arena now allocates at an offset past the first
    cond's still-live outputs, preventing MoveCall aliasing from corrupting data
  • Fix the same class of bug for nested torch.cond — propagate the parent submodule's input_mem_buffer_sizes down to nested HOPs so their branches don't allocate at
    offset 0 and overwrite the enclosing scope's live outputs
  • Refactor _apply_algo_to_submodules to process all HOP types (cond/while/map/scan) in a single graph-order pass with lifetime tracking, replacing the previous per-type
    iteration that had no cross-HOP awareness

Problem

ExecuTorch's MoveCall instruction does a shallow pointer move (not a data copy) to transfer a cond branch's output into the outer scope. If a subsequent cond's submodule
arena overlaps the first cond's output memory, executing the second cond overwrites data that the first cond's output pointer still references. This manifests as silent
data corruption at runtime.

Two variants:

  1. Sibling HOPs: Two torch.cond at the same level where cond_0's output is still live when cond_1 executes
  2. Nested HOPs: A torch.cond inside another cond's branch, where the inner branch allocates at offset 0 and overlaps with the outer cond's output

Fix

The rewritten _apply_algo_to_submodules now:

  1. Processes HOPs in graph order (single pass)
  2. Tracks each HOP's output lifetime via _hop_output_last_use
  3. When a later HOP's start index falls within a prior HOP's output lifetime, forces the later HOP's branches to allocate at an offset past the prior's memory
  4. Reads the parent's input_mem_buffer_sizes and merges it into the offset for nested HOPs
  5. Treats branches within the same HOP as mutually exclusive (merged via max, not stacked)

Test plan

  • test_overlapping_cond_outputs_no_corruption — sequential conds with overlapping output lifetimes, verifies runtime outputs against expected values across 4 stateful
    calls
  • test_non_overlapping_conds_share_memory — confirms non-overlapping conds still share memory (no unnecessary bloat)
  • test_nested_cond_no_corruption — nested cond inside a cond branch, verifies the inner branch doesn't corrupt the outer output
  • All 40 existing memory planning tests pass

Co-Authored by Claude Opus 4.6

@pytorch-bot

pytorch-bot Bot commented Jul 30, 2026

Copy link
Copy Markdown

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/21494

Note: Links to docs will display an error until the docs builds have been completed.

⚠️ 13 Awaiting Approval

As of commit d0f17ec with merge base b26b9ac (image):

AWAITING APPROVAL - The following workflows need approval before CI can run:

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@meta-cla

meta-cla Bot commented Jul 30, 2026

Copy link
Copy Markdown

Hi @gtyukasz!

Thank you for your pull request and welcome to our community.

Action Required

In order to merge any pull request (code, docs, etc.), we require contributors to sign our Contributor License Agreement, and we don't seem to have one on file for you.

Process

In order for us to review and merge your suggested changes, please sign at https://code.facebook.com/cla. If you are contributing on behalf of someone else (eg your employer), the individual CLA may not be sufficient and your employer may need to sign the corporate CLA.

Once the CLA is signed, our tooling will perform checks and validations. Afterwards, the pull request will be tagged with CLA signed. The tagging process may take up to 1 hour after signing. Please give it that time before contacting us about it.

If you have received this in error or have any questions, please contact us at cla@meta.com. Thanks!

@linux-foundation-easycla

linux-foundation-easycla Bot commented Jul 30, 2026

Copy link
Copy Markdown

CLA Not Signed

@pytorch-bot

pytorch-bot Bot commented Jul 30, 2026

Copy link
Copy Markdown

Didn't find following labels among repository labels: Release notes: exir

@gtyukasz

Copy link
Copy Markdown
Author

@pytorchbot label "release notes: exir"

@pytorch-bot pytorch-bot Bot added the release notes: exir Changes to any dialects and passes on these dialects, such as memory planning label Jul 30, 2026
…tput lifetimes

When multiple cond/while nodes exist in a graph and an earlier HOP's
outputs are still live when a later HOP executes, the later HOP's
submodule would overwrite the earlier HOP's results in the shared
arena starting at offset 0.

Rewrite _apply_algo_to_submodules to process HOPs in graph order,
track output lifetimes, and offset later HOP submodule allocations
past earlier HOPs' still-live memory. Map/scan outputs don't live in
the submodule arena so they are excluded from overlap tracking.
@gtyukasz
gtyukasz force-pushed the bugfix/higher-order-op-output-memplan branch from 040a60c to d0f17ec Compare July 30, 2026 15:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

release notes: exir Changes to any dialects and passes on these dialects, such as memory planning

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Higher Order Op Output Memory Planning

1 participant