Skip to content

Grid: the empty-digest quorum guard, and the instrument on the manifest - #33

Merged
JamesKane merged 2 commits into
mainfrom
fix/grid-followups
Aug 25, 2026
Merged

Grid: the empty-digest quorum guard, and the instrument on the manifest#33
JamesKane merged 2 commits into
mainfrom
fix/grid-followups

Conversation

@JamesKane

Copy link
Copy Markdown
Owner

Two AppView-side changes that came out of reviewing the Navigator edge
(decodingus-navigator#61).
Both are guards the AppView needs regardless of how well-behaved a node is.

Two empty digests must not form a quorum

A digest with no comparable call agrees with every other such digest — every field is
absent on both sides. a_shapeless_digest_agrees_with_nothing_real already asserted
exactly that and called it "equally uninformative", and I did not follow the thought
through: two nodes whose analysis failed would agree, canonicalize a unit with no
content, and be credited for it.

digest::has_content now answers whether a digest is about anything, and
grid-validate refuses to canonicalize an empty winning cluster. One discrete call
is enough.

The Navigator side was fixed in the same pass — a node now fails its unit rather than
submitting an empty digest. This guard is here anyway, because a node is untrusted
by construction and that is the whole premise adaptive replication rests on. A check
that only runs on the honest side is not a check.

The instrument on the curation manifest

A node cannot choose a mapper preset without it. A read set with no mate is either a
long read or a single-end short read, and those need different presets — a long read
mapped under a short-read preset does not fail, it produces alignments that look
right and are wrong
.

crawl_project already stores ENA's instrument model on the library, so this
publishes a value we already hold. Navigator feeds it to Preset::infer, which errors
rather than guesses on an unrecognised instrument, so the unit goes back to the pool
instead of a result of unknown quality reaching the quorum.

Without this, every single-end Illumina run in the catalogue would have been mapped as
HiFi.

Verifying

du-db's grid tests are live-Postgres and skip without DATABASE_URL. To run them:

PGHOST=$(container inspect du-testpg | python3 -c "import sys,json;print(json.load(sys.stdin)[0]['networks'][0]['ipv4Address'].split('/')[0])")
DATABASE_URL="postgres://postgres:dev@$PGHOST:5432/postgres?sslmode=disable" \
  cargo test -p du-db --test grid -- --test-threads=1

Serially, with ≥4 GB — each test migrates its own database.

CI

origin/main still fails cargo clippy --workspace -- -D warnings under current
stable clippy, in files this branch does not touch (du-db/src/place.rs:297
is_none_or vs the declared MSRV 1.80, plus du-web/src/origins_layout.rs:505,512).
Raised on #32 and still unaddressed; happy to fix in a separate PR if you want it.

🤖 Generated with Claude Code

https://claude.ai/code/session_01SequryftjcFkLnjQRMKCx3

JamesKane and others added 2 commits August 25, 2026 14:22
A digest with no comparable call agrees with every other such digest — every field
is absent on both sides. `a_shapeless_digest_agrees_with_nothing_real` already
asserted exactly that and called it "equally uninformative", and I did not follow the
thought through: two nodes whose analysis failed would agree, canonicalize a unit
with no content, and be credited for it.

`digest::has_content` now answers whether a digest is about anything, and the
validator refuses to canonicalize when the winning cluster is empty. One discrete
call is enough.

The Navigator side is being fixed in the same pass — a node now fails its unit
instead of submitting an empty digest. This guard is here anyway, because a node is
untrusted by construction and that is the entire premise adaptive replication rests
on. A check that only runs on the honest side is not a check.

Found by a review of the Navigator branch, which traced the consequence back across
the repository boundary.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SequryftjcFkLnjQRMKCx3
The node cannot pick a mapper preset without it. A read set with no mate is either a
long read or a single-end short read, and those need different presets — a long read
mapped under a short-read preset does not fail, it produces alignments that look
right and are wrong.

`crawl_project` already stores ENA's instrument model on the library, so this only
publishes a value we hold. The Navigator side then feeds it to `Preset::infer`, which
errors rather than guesses on an instrument it does not recognise.

Found by the third review of the Navigator branch: the node was choosing its preset
from the mate count, so every single-end Illumina run would have been mapped as HiFi.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SequryftjcFkLnjQRMKCx3
@JamesKane
JamesKane merged commit adb1576 into main Aug 25, 2026
1 check failed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant