Skip to content

fix(pipeline): resolve gap paths to real files - #479

Merged
kabraham13 merged 1 commit into
mainfrom
fix/docs-pipeline-gap-paths
Oct 28, 2026
Merged

kabraham13 merged 1 commit into
mainfrom
fix/docs-pipeline-gap-paths

Conversation

@kabraham13

Copy link
Copy Markdown
Contributor

Problem

The "Docs Pipeline" workflow fails on every push to main that touches api-reference/openapi.yaml. It failed on each of the last 8 runs, back to 2026-09-29, most recently on #478 (Run the docs pipeline step, exit code 1).

The log shows:

[11/11] Generating: thin_page for unknown...
  Error: [Errno 21] Is a directory: '.../docs/resources/breaking-changes-change-notices'
Done. Generated: 2, Errors: 1
No .mdx files found in pipeline/drafts/<run>
EVALUATE FAILED - drafts have errors.

Cause

detect_gaps.py takes page IDs from docs.json, which have no extension (docs/resources/foo), and puts them in the path of thin_page, missing_description and apiref_narrative_page gaps. The later steps treat path as a file:

  • generate.py reads REPO_ROOT / path. For a page ID that is not a file, load_file swallows FileNotFoundError and returns an empty string, so the model is asked to expand a page it was never shown. When a folder has the same name as the page (breaking-changes-change-notices.mdx and breaking-changes-change-notices/), the read raises IsADirectoryError and the run crashes.
  • The draft is named from the same path (docs--getting-started--supported-languages, no .mdx), so evaluate.py, which globs *.mdx, finds no drafts and fails the run.

evaluate.py and test_open_prs.py already assume extension-bearing paths (docs--admin--overview.mdx, docs/a.mdx).

Measured on current main: all 7 gaps that carry a path pointed at something that is not a file (5 with no extension, 2 that resolve to a directory).

Change

  • pipeline/util.py: add page_file(page_id), which returns the repo-relative .mdx or .md file for a page ID, or None.
  • pipeline/detect_gaps.py: use it where the three gap types are emitted. Detection is unchanged: the same 13 gaps, with only the path values differing.
  • pipeline/test_page_file.py: 6 stdlib tests, including a page next to a same-named folder and a folder with no page.

Verification

  • python -m unittest discover -s pipeline -p "test_*.py" passes (30 tests: 24 existing, 6 new), without yaml installed, as in CI.
  • Detector output on main before and after: 13 gaps both times, everything except path identical, and 0 of 7 paths are non-files after (7 of 7 before).
  • With the model client stubbed, generate.py now names the three thin_page drafts docs--...--<page>.mdx and load_file reads the real page content for each.

Not verified: a full pipeline run, which needs the model API and the later steps (review, promote, ship).

Heads up

Once this merges, the next change to openapi.yaml lets the pipeline get past generate and evaluate for the first time in these runs. It will call the model to expand the three thin pages (supported-languages, google-sheets, breaking-changes-change-notices) and may open a PR with the result. Use the manual workflow_dispatch with dry_run to see what it would do before relying on it.

🤖 Generated with Claude Code

detect_gaps.py copied docs.json page IDs, which have no extension, into
the path of thin_page, missing_description and apiref_narrative_page
gaps. The later steps expect a file:

- generate.py read REPO_ROOT / path, got FileNotFoundError (swallowed by
  load_file, so the model saw an empty page) or, when a folder shares the
  page's name, IsADirectoryError (the run crashed)
- the draft was written as docs--x--y with no .mdx, so evaluate.py found
  no .mdx files and the run failed

Add util.page_file() to resolve a page ID to its .mdx/.md file and use it
where the detector emits those three gap types. Detection is unchanged;
only the path value differs. Add a stdlib test for the helper.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
@mintlify

mintlify Bot commented Oct 28, 2026 •

Copy link
Copy Markdown
Contributor

Preview deployment for your docs. Learn more about Mintlify Previews.

Project Status Preview Updated
deepl-c950b784 🟢 Ready View Preview Oct 28, 2026, 6:10 PM

💡 Tip: Enable Automations to automatically generate PRs for you.

@kabraham13
kabraham13 merged commit 7294f6f into main Oct 28, 2026
4 checks passed
@kabraham13
kabraham13 deleted the fix/docs-pipeline-gap-paths branch October 28, 2026 18:20
shirgoldbird added a commit that referenced this pull request Oct 28, 2026
…mport

Carries over the parts of #450 that #479 didn't cover:

- detect_gaps: pages with child pages are exempt from the thin-page check.
  breaking-changes-change-notices is short because its content sits on its
  children; padding it out is the wrong fix.
- generate: load_existing_page() raises when the page a gap edits is missing
  or empty, so a failed read can't become "write this page from scratch".
- generate: import the Anthropic SDK only when a client is built, so
  --dry-run and the tests run without it. CI installs pyyaml for the new tests.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

This branch was successfully deployed

1 active deployment
staging — 2777eebe Deployed Oct 28, 2026 by mintlify[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants