Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
115 changes: 115 additions & 0 deletions .claude/skills/scheduling/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,115 @@
---
name: scheduling
description: >-
Guidance for scheduling Halide pipelines — deciding how a pipeline is
computed, stored, vectorized, and parallelized to run fast on a target,
without changing what it computes. Use this whenever the task involves
writing, reviewing, or optimizing a Halide schedule, or reaching for any
scheduling directive (compute_root, compute_at, store_at, store_root,
hoist_storage, vectorize, parallel, unroll, split, tile, fuse, reorder,
gpu_tile/gpu_blocks/gpu_threads, rfactor, compute_with, in/clone_in,
specialize, bound, ...). Also use it for reading `print_loop_nest` output,
diagnosing a slow or high-memory Halide pipeline, choosing inline vs
compute_at vs compute_root, or interpreting bounds inference. Prefer this
skill over guessing at directive semantics or schedule shape — it routes to
an in-repo guide with a chapter per directive. Trigger it even when the user
only says a Halide pipeline is "slow", mentions vectorization/parallelism/
tiling/sliding windows, or asks how a `Func` actually gets computed, without
saying "scheduling" explicitly.
---

# Scheduling Halide Pipelines

Halide splits every program into an **algorithm** (what each value is) and a
**schedule** (when and where each value is computed and stored). The schedule
can't change the result — it only moves three levers, and nearly all
performance comes from them:

- the **order** values are computed in,
- how much **redundant recomputation** happens, and
- how much **temporary storage** the pipeline needs.

This skill routes to a detailed in-repo guide rather than reproducing it. Read
the chapter a task actually needs *before* writing or changing a schedule: the
directives have precise, easy-to-misuse semantics (`store_at` and `compute_at`
control storage and compute *independently*; a misplaced `parallel` can
serialize, race, or explode memory; `For` loops carry an inclusive `min`/`max`,
not `min`/`extent`). Confirm the result with `print_loop_nest()` rather than
assuming.

## Start here

Read the landing page first — it has the full table of contents and the
four-part structure:

- **`references/guide/README.md`**

For a fast, one-screen lookup — every directive's signature and effect, the tail
strategies, and the common gotchas — without opening a chapter, read the bundled
**`references/directive-cheatsheet.md`** (alongside this skill).

The chapters live under **`references/guide/`**. The guide and cheat-sheet paths
below are relative to this skill; the `src/`, `tutorial/`, and `python_bindings/`
paths are relative to the repository root.

## Which chapter to read

Jump to what the task needs instead of reading front to back.

**The model** (what is being scheduled)

- `references/guide/01-introduction.md` — algorithm vs schedule; the three levers.
- `references/guide/02-the-programming-model.md` — Funcs, stages, RDoms, the graph.
- `references/guide/03-realizing-a-pipeline.md` — when computation runs; JIT vs AOT.
- `references/guide/04-bounds-inference.md` — how buffers and loops get sized.

**Making it fast** (start here for "make this faster")

- `references/guide/05-scheduling-for-cpus.md` — the canonical fast-CPU shape.
- `references/guide/06-scheduling-for-gpus.md` — mapping loops onto blocks/threads.
- `references/guide/07-what-to-schedule.md` — inline vs compute_at vs compute_root.
- `references/guide/08-benchmarking-and-profiling.md` — measure; read the profile.
- `references/guide/09-reading-the-stmt-file.md` — check the vectorization shape.
- `references/guide/10-recipes.md` — sliding windows, tiling, pyramids, stencils, histograms.
- `references/guide/11-pitfalls.md` — recurrences, parallel placement, recompute traps.

**Directive mechanics** (what a directive does to the loop nest)

- `references/guide/12-reading-a-loop-nest.md` — the `print_loop_nest` notation.
- `references/guide/13-defaults-and-inlining.md` — the inline default; three compute levels.
- `references/guide/14-placement-compute-root-and-compute-at.md` — where a Func is built.
- `references/guide/15-storage-levels.md` — `store_at`, `store_root`, `hoist_storage`.
- `references/guide/16-reshaping-loops.md` — `split`, `fuse`, `reorder`, `tile`.
- `references/guide/17-loop-types.md` — serial, parallel, vectorized, unrolled, GPU.
- `references/guide/18-advanced-directives.md` — `rfactor`, `in`/`clone_in`, `compute_with`, `specialize`.
- `references/guide/19-how-the-loop-nest-is-built.md` — the full assembly order.

**Reference**

- `references/guide/20-directive-reference.md` — every directive, its signature and effect.
- `references/guide/21-checklist-and-worked-example.md` — pre-flight checklist + end-to-end example.

## Inspecting a schedule

- `Func::print_loop_nest()` prints the loop nest a schedule produces — the
fastest way to confirm it did what you meant. Chapter 12 explains the notation.
- `HL_DEBUG_CODEGEN=1` (raise for more detail) prints the IR after key lowering
passes; `HL_JIT_TARGET` / `HL_TARGET` set the target being scheduled for.
- The built-in profiler and the `.stmt` file (chapters 8–9) show where time goes
and whether the hot loop is a unit-stride vector store.

## Related material in this repo

- **Tutorials:** `tutorial/lesson_05_scheduling_1.cpp`,
`tutorial/lesson_08_scheduling_2.cpp`, `tutorial/lesson_12_using_the_gpu.cpp`,
`tutorial/lesson_18_parallel_associative_reductions.cpp`.
- **Scheduling API:** directive declarations live in `src/Func.h` and
`src/Schedule.h`; the passes that consume a schedule are in
`src/ScheduleFunctions.cpp`, `src/Bounds.cpp`, and `src/BoundsInference.cpp`.
- **Autoschedulers** (let Halide propose a schedule via
`Pipeline::apply_autoscheduler`): `src/autoschedulers/` — `mullapudi2016`,
`adams2019`, `li2018`, `anderson2021`.
- **Python bindings:** the guide's examples are C++, but the scheduling API is
identical in Python — the same directives are methods on `halide.Func`, so a
C++ schedule maps over directly. Python tutorials mirror the C++ ones under
`python_bindings/halide/tutorial/`.
82 changes: 82 additions & 0 deletions .claude/skills/scheduling/references/directive-cheatsheet.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,82 @@
# Halide scheduling directive cheat sheet

A compact, one-screen lookup. Each directive is a method on `Func` (or, for an
update stage, `Func::update(int)`). For the full prose reference and examples,
see `guide/20-directive-reference.md`; the group headings below note the chapter
that explains the mechanics.

## Where to compute and store — ch. 13–15

| Directive | Effect |
| ------------------------------- | -------------------------------------------------------------------------------- |
| `f.compute_root()` | Compute all of `f` once, before any consumer. |
| `f.compute_at(g, var)` | Compute the needed slice of `f` inside `g`'s `var` loop. |
| `f.compute_inline()` | Reset to the inline default (undo a `compute_*`). |
| `f.store_at(g, var)` | Allocate `f` at `g`'s `var` loop; coarser than compute enables a sliding window. |
| `f.store_root()` | Allocate `f` at the outermost level. |
| `f.hoist_storage(g, var)` | Hoist only the allocation to `g`'s `var` loop. Never past a parallel loop. |
| `f.fold_storage(var, K)` | Circular buffer of `K` slots in `var` (sliding windows). |
| `f.store_in(MemoryType::Stack)` | Allocate on the stack; small fixed sizes only. |

## Loop shape — ch. 16

| Directive | Effect |
| -------------------------------------- | -------------------------------------------------------------------------- |
| `f.split(x, xo, xi, factor)` | Split `x` into outer `xo`, inner `xi` (`x = xo*factor + xi`). |
| `f.fuse(a, b, t)` | Collapse two adjacent loops into one `t`. |
| `f.tile(x, y, xo, yo, xi, yi, tx, ty)` | Two splits plus a reorder into tiled traversal. |
| `f.reorder(a, b, c, …)` | Reorder loops, innermost first (last arg = outermost). |
| `f.reorder_storage(a, b, …)` | Change memory layout, innermost dim first. |
| `f.bound(x, min, extent)` | Promise `x`'s range at compile time (enables fixed-size unroll/vectorize). |

## Loop types / execution — ch. 17

| Directive | Effect |
| ------------------------------------------------ | ----------------------------------------------------------------------------- |
| `f.parallel(var[, task_size])` | Run `var` on the thread pool; `task_size` blocks it into fewer, larger tasks. |
| `f.vectorize(var[, factor])` | Emit SIMD; prefer `factor = natural_vector_size<T>()`. |
| `f.unroll(var[, factor])` | Unroll the loop (stays scalar). |
| `f.serial(var)` | Reset a dimension to serial. |
| `f.gpu_tile(…)`, `f.gpu_blocks/threads/lanes(…)` | Map dimensions to GPU blocks/threads/lanes. |

## Wrappers, reductions, variants — ch. 18

| Directive | Effect |
| ----------------------------------------------- | -------------------------------------------------------------- |
| `f.in(g)` | Identity wrapper of `f` for consumer(s) `g`. |
| `f.clone_in(g)` | Fresh recomputed copy of `f` for `g`, scheduled independently. |
| `f.update(n).rfactor(rv, v)` | Factor a reduction so the preserved axis parallelizes. |
| `b.compute_with(a, var)` | Fuse two stages into one shared loop nest down to `var`. |
| `f.specialize(cond)` / `f.specialize_fail(msg)` | Run-time-selected schedule variant / hard fallback. |

## Tail strategies — passed to a split (or the factor forms of vectorize/unroll)

| Strategy | Behavior |
| ---------------------------- | ---------------------------------------------------------------------------------- |
| `TailStrategy::RoundUp` | Fastest; overshoots and does extra work. Needs the producer valid past its extent. |
| `TailStrategy::ShiftInwards` | Default; overlaps the last tile with the previous one. Safe for pure Funcs. |
| `TailStrategy::GuardWithIf` | Adds an if-check; slowest, always safe. |

## Diagnostics

| Tool | Use |
| -------------------------------------------- | --------------------------------------------------------------------------- |
| `f.print_loop_nest()` | Print the loop structure; check placement and vectorization shape (ch. 12). |
| `HL_TARGET=host-profile` / `Target::Profile` | Per-Func profiler table — the primary diagnostic (ch. 8). |
| `HL_DEBUG_CODEGEN=1` | Dump IR after key lowering passes (raise the number for more detail). |

## Gotchas

- **`For` loops carry an inclusive `min`/`max`**, not `min`/`extent` — a common
source of apparent off-by-one errors.
- **`reorder` lists innermost first**; the last argument is the outermost loop.
- **`store_at` and `compute_at` are independent** — storing coarser than you
compute is exactly what enables sliding-window reuse.
- **Don't `store_at`/`hoist_storage` past a `parallel` loop** that would race on
the shared allocation.
- **A serial recurrence can't be parallelized or vectorized** along the
dependent axis — see ch. 11.
- **`RoundUp` needs the producer valid past its extent** (e.g. via a boundary
condition); otherwise keep the default `ShiftInwards`.
- **Vectorize width should be `natural_vector_size<T>()`**, not a guessed
constant.
66 changes: 66 additions & 0 deletions .claude/skills/scheduling/references/guide/01-introduction.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
# 1. Introduction

Every Halide program comes in two parts, written separately, and keeping them
apart is central to how the language works.

The first part is the **algorithm**. It says what each value is, and nothing
about how that value is computed. A program declares `Func`s and defines them as
pure functions of their argument `Var`s, with optional *update definitions* that
refine those values. At this stage the description is pure math, with nothing
said about execution.

The second part is the **schedule**. It says when and where each of those values
gets computed and stored. The schedule is written as directives (`compute_root`,
`compute_at`, `vectorize`, and so on) attached to each `Func`, and it's where
all of the performance decisions live.

What makes this split so useful is that the schedule can't change the answer.
For a given algorithm and input, the result is the same no matter how it's
scheduled, apart from small floating-point effects like round-off or overflow.
That leaves a lot of room to maneuver: the computation can be rearranged
aggressively in the name of speed, with no risk of quietly changing what the
program computes.

So if the schedule leaves the result alone, what does it actually affect? It
really comes down to three things:

- the **order** values get computed in,
- how much **redundant recomputation** happens, and
- how much **temporary storage** the pipeline needs.

Those three levers are where nearly all of the performance comes from. The gap
between a careless schedule and a careful one can be orders of magnitude, and a
poorly chosen schedule can even fail to compile, all while computing exactly the
same thing.

## What the schedule controls

For each `Func`, and for each of its stages, the schedule decides a few things:

- **Where it's computed.** Inlined into its consumer, computed at some loop
level of a consumer, or computed once at the top (what Halide calls "root").
- **Where its storage lives.** This can sit at a coarser loop than where the
values are computed, which is exactly what lets a sliding window reuse them.
- **How its loops are shaped.** They can be split, tiled, and reordered, and
marked parallel, vectorized, or unrolled.
- **How its storage is laid out** in memory.

Later chapters take these one at a time. For now the takeaway is that scheduling
is a handful of independent choices, made per Func.

## What this manual covers

**Part I** lays the groundwork: the objects being scheduled, how a pipeline is
realized, and how Halide infers the bounds of every buffer and loop.

**Part II** is the **strategy**: given a pipeline, which schedule is worth
writing, and how to spot what to fix. It's the fast path to a schedule that runs
well.

**Part III** is the **mechanics**: given a schedule, what loop nest does it
actually produce? It opens by reading `Func::print_loop_nest()` output, then
works through each directive one at a time. When Part II reaches for a
directive, its precise effect lives here.

**Part IV** is a compact reference: a directive index, a checklist, and a worked
example.
Original file line number Diff line number Diff line change
@@ -0,0 +1,89 @@
# 2. The Programming Model

Scheduling works on a handful of objects. This chapter names them and shows the
graph they form. The rest of the manual assumes this vocabulary.

## The objects

**`Var`.** A name for a dimension, which is a loop variable. A `Var` holds no
state beyond its own identity. Vars are the arguments of a pure definition, and
the handles named later in scheduling calls.

**`Func`.** A *handle* to a function definition. The `Func` is the thing that
gets scheduled. Copying a `Func` produces another handle to the same underlying
function, so scheduling through either one affects that single function. A
Func's state includes:

- its **name**, used for printing and for breaking ties in computation order,
- an ordered list of **stages**: an initial (pure) definition plus zero or more
update definitions (below),
- the set of **other Funcs it reads from**, its *producers*, which come from its
definitions,
- its **compute level** and **store level**: where it's computed and where its
storage lives. The default is *inline*.

**`Expr`.** A value expression on the right-hand side of a definition. For the
loop nest, the only thing that matters about an `Expr` is which Funcs it reads.
That's what wires up the producer/consumer graph. Plain arithmetic, constants,
and `cast<T>(...)` never show up in the loop nest. They live inside the leaf
`f(...) = ...` line.

**`ImageParam`.** An input buffer. It's a *leaf*. It never gets computed and
never shows up in the loop nest. A Func that reads an `ImageParam` just has one
fewer producer to build. The buffer is already there.

**`RDom` / `RVar`.** A *reduction domain* and its *reduction variables*. An
`RDom r(min, extent, …)` declares one or more `RVar`s (`r.x`, `r.y`, and so on)
for use in update definitions. An `RVar` names a loop like a `Var` does. The
difference: its iterations are *ordered*, and can carry a dependency from one to
the next, like a running sum. RVars only appear in update definitions.

## Stages and update definitions

A Func can have **update definitions**. These are assignments, written after the
first one, that change the Func's values in place.

```cpp
Func hist("hist");
hist(x) = 0; // stage 0: the pure (initial) definition
RDom r(0, N, "r");
hist(in(r)) += 1; // stage 1: an update definition
```

The initial definition plus the updates form an ordered list of **stages**: `s0`
(pure), `s1`, and so on. Each stage finishes before the next one starts. They
all write to the same storage. Stages are part of the algorithm. They say what
the Func is, so no scheduling is needed to understand them. Each one is still
scheduled on its own, with `f.update(n)` (see
[Reshaping Loops](16-reshaping-loops.md)).

An update definition can reference **at most one** `RDom`. Several reduction
axes come from a single multi-dimensional `RDom`, like `RDom r(0, 4, 0, 5)`.

## Pure vs non-pure

A Func is **pure** if it has only its initial definition, with no updates. Add
one or more update definitions and it's **non-pure**. This split matters all
through Part III.

A pure Func can be substituted into its callers as an expression. A non-pure
Func can't, because a reduction isn't an expression, so it has to be built into
storage. That one fact drives several rules ahead.

## The pipeline graph

The Funcs, connected by producer edges, form a directed acyclic graph. That's
the *pipeline*. One Func is the **output**: the one that `print_loop_nest()` (or
`realize`) is called on. Everything reachable from the output by following
producer edges is part of the pipeline. Nothing else is.

A couple of things are worth locking in early:

- The output is special. It's always computed once, at the top, and it never
gets inlined away.
- Forcing a Func to be non-inline makes it **opaque to bounds analysis**. Take
an affine helper like `helper(x) = m*x + b`, used as an index in
`f(helper(x))`. Inlined, it compiles fine. Force it non-inline, though, and
bounds inference can fail with
`calls ... in an unbounded way in dimension ...`. So it's usually best to keep
index helpers inlined.
Loading
Loading