Skip to content

docs: define a currency policy for docs/dev/ design notes (several are stale/misleading) #1482

Description

@planetf1

Problem

docs/dev/ holds internal design-rationale notes that sit outside the published documentation site, but are linked from several docs/examples/*/README.md files and cited directly from source-code docstrings. None carries any marker of whether it reflects current behaviour, a superseded design, or an unresolved brainstorm — a reader, human or coding agent, has no way to tell without independently checking the source.

This is not hypothetical. #1464 documents a case where a docs/dev note (adapter_observability.md, written for the then in-progress #1141) named start_backend_span/start_action_span as the pattern to mirror — the exact direct-span-call approach that #1181 had deliberately replaced with a hook-and-plugin model six weeks earlier. An implementer followed the note as written, produced spans directly from mellea/backends/, and the change had to be reverted in PR #1454. The published page (docs/docs/observability/tracing.md) never mentioned the hook/plugin rule either, so there was no correct reference to fall back on — the stale internal note was the only guidance available.

Checking the notes currently on main against today's code shows the same pattern at smaller scale:

File Status Evidence
generate_ctx_signature.md Current Matches Backend.generate_from_context(self, action, ctx, ...) as implemented
hook_system.md Current Matches the mellea/plugins/ subsystem; has a published counterpart in docs/docs/concepts/plugins.mdx
requirement_aLoRA_rerouting.md Current Its Phase 1 section's self._find_adapter(name, ("alora",)) call matches mellea/backends/huggingface.py and mellea/backends/openai.py verbatim
intrinsics_and_adapters.md Stale, already tracked Describes the old four-class adapter hierarchy; #1144 already schedules a full rewrite
tool_calling.md Stale Proposes SessionToolCallingProtocol/MelleaSessionToolProtocol with tool_call_result()/call_tool() — none exist in the codebase; the example it's linked from uses an unrelated ad hoc design instead
mify.md Stale Example instantiates a class with no __init__ via positional arguments (would raise); points at a module path that moved
spans.md Stub Twenty lines, ends mid-thought; no SpanContext or __span_repr__ exists in the codebase; still linked from an example README as though it documents settled architecture
constrained_decoding.md Stub Informal open questions, no recorded decision, not linked from anywhere
mellea_library.md Directionally accurate, unverifiable A library-vs-framework philosophy statement, still broadly consistent with the codebase but written as fragments rather than checkable claims

Consequences of leaving this unaddressed:

  • A contributor or agent follows a linked dev note and writes code against an API that does not exist, or reintroduces a pattern the codebase deliberately moved away from — as already happened.
  • The accurate notes lose credibility by association once a reader has been burned by a stale one nearby.
  • Nothing today prevents the next note from rotting the same way.

Requirements

  1. A reader must be able to tell, without leaving the file, whether a docs/dev/ note reflects current behaviour.
  2. A note that no longer matches the code it describes must not continue being linked from example READMEs without qualification.
  3. Every cross-reference to a docs/dev/ file — from source comments, generated docs, or example READMEs — must resolve to a file that actually exists.
  4. A note that never reached a decision must be visibly distinguishable from one that records a settled decision.
  5. Design exploration that stalls must still have somewhere to live — the policy must not force premature deletion of unfinished thinking, only require it to be labelled as such.
  6. Whichever mechanism is chosen must not silently rot again — either it is self-maintaining, or someone is explicitly on the hook for keeping it current.

Proposed starting point (not a decision — open to challenge)

  • Resolve directly out of this discussion rather than spawning a follow-up issue per note. Two files already have dedicated tracked fixes — intrinsics_and_adapters.md via refactor(intrinsics): remove deprecation shims; rewrite intrinsics_and_adapters.md; write 3 tutorials (Epic #929 Phase 4) #1144, adapter_observability.md via docs(telemetry): document the hook/plugin span-production rule and enforce it (Epic #929) #1464 — everything else can be marked, corrected, or retired as agreement is reached here, with PRs opened against whichever notes need changing. Alternative: still open one issue per note for a clearer audit trail — more process overhead than the remaining fixes likely justify.
  • Mark, don't necessarily rewrite, notes not yet verified against current code, with a short visible caveat at the top. Alternative: rewrite immediately — resolves faster, but ties this issue to writing time rather than getting a lightweight signal in place quickly.
  • Retire notes with no external links and no recorded decision. Alternative: keep them as a record of design exploration — cheap, but only if there's an agreed place they're expected to be read from, which this review found none of.
  • Add a short paragraph to AGENTS.md stating that docs/dev/ notes are not guaranteed current and should be checked against source before being relied on or cited. Alternative: rely on per-file caveats alone — less visible before a specific file is opened, but avoids adding another rule to an already-large document.

Explicitly not proposed here: no change to the published documentation site, and no change to which subsystems get a docs/dev/ note in the first place.

Testing and documentation

Documentation-only; no code paths are affected by this issue itself. Whatever is agreed should be checked by:

  • the existing markdownlint gate, applied to any file a follow-up touches
  • a link-check pass confirming every docs/dev/*.md reference (from source comments and example READMEs) resolves to a file that exists — this review found one that currently does not (generate_signature_decisions.md, referenced in three source files and two generated docs; the real file is generate_ctx_signature.md)

Related: #1144 (rewrite of intrinsics_and_adapters.md), #1464 (the hook/OTel incident and its fix), #1433 (broader agent-enablement epic this may feed into).

Metadata

Metadata

Assignees

No one assigned

    Labels

    documentationImprovements or additions to documentationneeds-designProblem is clear; implementation approach needs design discussion before work startsp2Medium/low: minor bugs, niche features, polish, docs, tests, cleanup. Scoped, lower urgency.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions