Skip to content

Check bibliography entries in reviews, and a reference check - #11

Merged
cu1ch3n merged 7 commits into
mainfrom
reference-check
Oct 31, 2026
Merged

cu1ch3n merged 7 commits into
mainfrom
reference-check

Conversation

@cu1ch3n

@cu1ch3n cu1ch3n commented Oct 31, 2026 •

Copy link
Copy Markdown
Member

Why

An agent can add a .bib entry for a work that does not exist, or with the wrong authors, year or DOI. Until now a .bib change showed up in the review as a truncated word diff of source lines, or only as the typeset bibliography line of a cited entry, and nothing was checked.

What changes

Reviews (review engine)

  • A .bib change is shown entry by entry: every field, old → new for modified fields, the lines it occupies, and where the paper cites it. Hunks of one entry are decided together, so an entry is never half accepted. Changes to @string definitions, including nested macros, show the old and new expanded fields. Bibliography files share macros in declaration order, so a definition change also checks dependent entries in unchanged files. Dependent hunks in one file are grouped; when multiple files change, every affected edit displays the entry and has the acceptance guard.
  • Added and modified entries are looked up online: DOI via doi.org, arXiv id via arXiv, otherwise title + authors in Crossref, DBLP and arXiv. Each entry is Verified, Differs from the record (fields side by side, with a link), Not verified, or Could not check. Results are cached in .git/latex-review. Cache keys and manual confirmations include the entry type, so changing an article into a book triggers a new check.
  • Flagged without any lookup: removed entries the paper still cites, duplicate keys, a .bib file the paper does not read, new \cite keys without an entry, including in documents that use \nocite{*}.
  • Accepting a change with an entry that is not verified asks first; Accept all leaves such changes for the author. I checked it myself marks an entry as checked by hand until its looked-up fields change.

Papilot

  • The review gateway and the uncommitted-changes view pass the new routes through; confirmations are shared through the repository.
  • Settings → Check references online (on by default; only titles, authors, years and identifiers are sent).
  • Agents are told never to write a bibliography entry from memory and to include its DOI or arXiv id.
  • More under Manuscript in the sidebar opens a submenu (a sheet on phones) with Reference check: the whole bibliography on disk — citations without an entry, duplicate keys, the same work under two keys, uncited entries, unreadable entries, and an online check of every entry. Ask an agent to look into opens an editable new task that asks the agent to change nothing and find out what happened with each problem (does the work exist, what does its record say, where did the entry come from in Git).

How it was checked

  • Papilot build and typecheck; review engine build and typecheck.
  • Application reference/proxy, changes and review-state checks pass (9 passed, 1 skipped); all 155 review-engine tests pass, including 35 bibliography/checker tests.
  • Four bibliography browser tests cover entry display, guarded acceptance, visible macro-expansion changes, and acceptance guards on every edit affected by cross-file macros. A real TeX/BibTeX regression verifies cross-file macro expansion and citation wrappers/code examples. Regression tests cover nested macro edits and partial acceptance, missing citations with \nocite{*}, type-sensitive caching, old cache invalidation and manual confirmations.
  • Papilot's review-flow browser checks pass, covering inline commits, answer-only tasks, acceptance and undo, review history, and mobile layouts.

Known issues

  • scripts/check-ui.mjs:524 (focus after collapsing the manuscript panel) is flaky: it failed once in two runs on a clean origin/main too. Engine browser tests review.spec.ts:7 and :359 also fail on main.
  • DBLP currently answers every client with a bot check, so it is skipped and entries say so; checks rely on Crossref, doi.org and arXiv.
  • Several entries inserted as one contiguous block stay one change: accepting them one by one would conflict in the three-way merge.
  • The online-check switch is stored per browser.

TODO

Reference check page

  • Group problems by kind (missing citations, differs, not found, duplicates, uncited) in collapsible sections; today "not found" books and reports make one long list.
  • Show progress as "n of m checked" with a spinner on each entry; tidy the summary line when it wraps.
  • Jump from a citation place to the PDF (SyncTeX forward) and from an entry to its .bib lines.
  • "I checked it myself" for several selected entries at once.
  • Bring the agent's findings back onto the page, entry by entry, instead of only in the task's chat.
  • Say "DBLP could not be reached" once for the whole check rather than under every entry.
  • One verb for the action across the review, the page and Settings (检查 / 核对 are mixed).

Review cards

  • Fold long author lists and long fields, with "show all".
  • Fold the unchanged fields of a modified entry.
  • Polish the phone layout of the field table, the record comparison and the confirm dialog.

Sidebar

  • Decide what else belongs under More (LaTeX warnings, consistency?) and whether the closed row shows a count.
  • Revisit the icon and the active state of the More row while the check page is open.

Sources and tests

  • A book source (Open Library or Google Books) to cut false "not found" for books.
  • A UI check for the More menu and the reference check page in scripts/.

🤖 Generated with Claude Code

An agent can add a .bib entry for a work that does not exist, or with
the wrong authors, year or DOI, and the review showed such a change as
a word diff of source lines, cut at 6000 characters, or only as the
bibliography line BibTeX typesets for a cited entry.

A round now reads the .bib files before and after and shows each entry
it adds, removes or modifies whole: every field, a modified field's old
and new value, and where the paper cites it. Hunks that touch the same
entry, or the same key on either side, make one change, so an entry is
never half accepted. A removed entry the paper still cites, a key
defined twice, a .bib file the paper does not read, and new citations
of keys without an entry are flagged. A .bib edit no longer takes the
"no visible change" shortcut, which treated % in a URL as a comment.

After a round is published, the page asks the engine to look up the
added and modified entries: a DOI through doi.org, an arXiv identifier
through arXiv, and otherwise the title and authors in Crossref, DBLP
and arXiv. Each entry is verified, differs from its record (with the
fields side by side), not verified, or could not be checked. Searches
pick the record that agrees in title, first author and year, so later
copies of a preprint or a review of a book are not taken for the work;
a preprint's year before the published version's and a subtitle are
small differences only. Results are kept in .git/latex-review. Only
titles, authors, years and identifiers are sent, one request at a time
per host, and a host that turns clients away is skipped and named.

Accepting a change with an entry that is not verified, or one with any
of the problems above, asks first; Accept all leaves such changes for
the author. An entry the author checked by hand can be marked so; the
confirmation holds until its looked-up fields change. The page skips
online checks when latex-review:references is "off" in local storage.

Real Crossref, doi.org and arXiv answers were checked on fabricated,
misattributed and correct entries; DBLP currently answers every client
with a bot check, so the tests use recorded answers only.
Papilot passes the engine's bibliography checks through: the review
gateway allows a round's references and the confirmation of an entry,
without refreshing the task or drafting a commit message for it, and
the review of uncommitted changes gets the same routes. Both share one
result file in the data folder, and confirmations live beside the
engine's state in the repository, so a task's review, the uncommitted
changes and the reference check agree. Settings gains "Check
references online", on by default, which the review page reads.

Agents are told never to write a bibliography entry from memory, to
take its details from the published record with its DOI or arXiv
identifier, and to name the entries they add or change.

More, under Manuscript in the sidebar, opens a menu beside it (a sheet
on a phone) with Reference check: the whole bibliography as it is on
disk, read only once the page is opened. Without looking anything up
it lists citations of keys no entry defines, keys defined twice, the
same work under two keys, uncited entries and entries BibTeX cannot
read; Check online compares every entry with its record. An entry can
be marked as checked by hand. Ask an agent to look into the problems
opens a new task, editable before it is sent, that asks the agent to
change nothing and find out what happened: whether each work exists,
what its record says, and where the entry or citation came from in Git.

The side menu also opens with the right arrow and closes with the
left, as a submenu does.
@cu1ch3n
cu1ch3n merged commit 7a5ec63 into main Oct 31, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant