Skip to content

validate-doc-claims.py resolves citations against the doc's own repo, so a learnings store in a submodule flags every path in the parent #1725

Description

@therealDimitri

skills/ce-compound/scripts/validate-doc-claims.py derives the repository it checks against from the document's own location:

code, repo_root = git(["rev-parse", "--show-toplevel"], doc_dir)   # :266 in 3.26.3

Every later path and SHA check then runs with cwd=repo_root, so the only tree the checker can see is the one the document sits in, and there is no argument to point it elsewhere.

That is correct when the store and the code share a repo. It breaks on a common layout: the learnings store is a git submodule of the repo whose code it documents. A doc that cites src/lib/foo.ts is checked against the docs repo, where no such path exists, so every citation comes back as:

FLAG path `src/lib/foo.ts` (line 39) - not found in working tree or origin/main.
Fix the citation, or annotate it as historical (e.g. removed by this fix).

and a commit from the parent repo comes back as:

FLAG sha abc12345 (line 35) - does not resolve to a commit in this repository.

Every one of those is a false positive, and the remedy each suggests is the wrong action, because the citations are right.

Why it matters. The signal inverts. Across three consecutive runs against one such store, every path flag was a false positive, which trains the reader to skim the output rather than act on it. That is the state in which a real flag gets missed.

Reproduction

git init code && cd code && mkdir -p src/lib
echo 'export const x = 1' > src/lib/foo.ts
git add -A && git commit -qm init

git init ../docs-store && cd ../docs-store && mkdir -p solutions
git commit -q --allow-empty -m init

cd ../code && git submodule add ../docs-store docs && git commit -qm 'add docs submodule'
printf -- '---\ntitle: x\n---\n\nSee `src/lib/foo.ts`.\n' > docs/solutions/x.md

python3 validate-doc-claims.py docs/solutions/x.md
# FLAG path `src/lib/foo.ts` ... not found. It exists, one directory up.

Suggested fix. When repo_root is a submodule, try the superproject before flagging:

code, sup = git(["rev-parse", "--show-superproject-working-tree"], repo_root)
# non-empty => also check candidate paths (and SHAs) against `sup`, and treat a hit there as found

--show-superproject-working-tree returns empty for a normal repo, so the extra call is a no-op in the common case.

Version: 3.26.3. The same line is present unchanged in 3.25.0, so this is not a recent regression.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions