Catalog push accepts a term that alias folding then makes invalid by legendko · Pull Request #47 · RAMP-Protocol/protocol

legendko · 2026-09-02T14:43:01Z

Why

A token cannot be both permitted and prohibited, and the contract says so as a CEL rule
over the request exactly as received. That rule compares spellings where the meaning
is a token. Ten restriction tokens have more than one accepted spelling — scrape is
a registered alias of crawl, adapt and derivative both mean modify, personal
means individual — and every axis also folds ASCII case. A term naming one spelling
under permitted and another under prohibited therefore passed the boundary check,
because as written the two lists shared nothing, and folded into a stored term with the
same token in both lists.

Nothing looked at it again. The stored term rides on offers and the Exchange validates
its own responses, so the failure surfaced at a different party, on a different RPC, an
unbounded time later: every discovery request returning that resource answered with an
internal error, while the push that caused it had been answered as accepted. It stayed
broken until someone edited the database.

No bad intent is needed. scrape and crawl are the AIPREF and RSL names for one thing,
and the protocol advertises both.

Decisions worth reading before the code

Both rules fire; neither suppresses the other. On the plain permitted:[x] prohibited:[x] case ValidateResourceEntry now reports the wire rule and the ingest rule, because it composes both tiers. Three precedents say report-alongside (design-history.md:1209 — "reported alongside the others rather than instead of them"; 26 multi-rule cases already in conformance/corpus/cases.json; the entry face is documented to over-report so a publisher fixes everything in one round). The deciding argument is narrower: connectserver defaults to ValidationOff, so on a mount that never asks for the wire tier the ingest check is the only tier a pushed term meets, and a rule that stayed silent because "the boundary caught it" would catch nothing there — reopening the bug for exactly the deployments least able to notice.
The check canonicalises internally, rather than trusting the term to have been normalised. Both callers do normalise, but a rule written to close an ordering gap must not open one of its own, and the fold is a fixed point so it costs a pass over already-canonical tokens. It is also what lets the per-term corpus list — which feeds terms as authored — express the fault at all.
It runs on every axis, including RESTRICTION_KIND_OTHER, where the fold is a no-op and the check is plain equality. Not gated on hasCanonicalRule; that gate is what the ValidationOff argument above rules out.
The finding names the canonical token, not the spellings that produced it. A message quoting scrape and crawl differs depending on whether the caller folded first — the Exchange has, a direct caller has not — and the message is wire-visible (CatalogRejection.Detail) and pinned byte-for-byte across three SDKs. Path locates the permitted element; the prohibited one is the entry that folds to the same token.
Its own rule id, not the CEL rule's. One name for two rules is what conformance/licenseterm_rule_ids_test.go forbids, and these genuinely are two rules: they read different values and can disagree.
The bug is wider than the ticket frames it. The fold case-folds per axis (lower for FUNCTION/USER_TYPE, upper for GEOGRAPHY), so permitted:["us"] with prohibited:["US"] collides on GEOGRAPHY — an axis with zero aliases. Same rule, more vectors.
The sibling rule is unaffected, and now says so. NormalizeLicenseTerm rewrites tokens and never kind, so license_term.one_restriction_per_kind cannot be created or destroyed by folding. That was assumed by the tier split; a unit test now asserts it.

A token cannot be both permitted and prohibited, and the contract says so as a CEL rule over the request exactly as received. That rule compares SPELLINGS where the meaning is a TOKEN. Ten restriction tokens have more than one accepted spelling — scrape is an alias of crawl, adapt and derivative both mean modify, personal means individual — and every axis also folds ASCII case, lower for FUNCTION and USER_TYPE, upper for GEOGRAPHY. So a term naming one spelling under permitted and another under prohibited passes the boundary check, because as written the two lists share nothing, and then folds into a stored term where the same token sits in both lists. Nothing looked at it again. The stored term rides on offers and the Exchange validates its own responses, so the failure surfaced far away and much later: every discovery request returning that resource answered with an internal error, the push having looked clean and the publisher having been told nothing. The ingest tier now asserts the same property over the canonical values, under its own rule id — one name per rule, which a conformance guard requires. The boundary rule is unchanged: it still catches the plain case, and it costs nothing. Three properties are deliberate. The check folds what it compares instead of trusting the term to have been normalised. Both callers do normalise — the Exchange in place, the entry face on a copy — but a rule written to close an ordering gap must not open one of its own, and the fold is a fixed point, so it costs a pass over tokens already folded and nothing else. It is also what lets the corpus express the fault at all, since the per-term list feeds terms as authored. It runs on every axis, including the one with no canonicalisation rule. There the fold returns the token unchanged and the comparison is plain equality, which is what it must be: the server binding defaults to validation off, and on such a mount the ingest tier is the only tier a pushed term meets. The finding names the canonical token and not the spellings that produced it. A message quoting the spellings differs depending on whether the caller folded first, and the message is wire-visible and pinned byte for byte across three SDKs. The path locates the permitted element; the prohibited one is the entry that folds to the same token. The per-alias vectors are derived from the generated vocabulary rather than listed, so an alias added to the proto brings its case with it. The entry corpus now records the plain overlap under both rules, which is what composing the two tiers means: the boundary reports what it sees, the ingest tier reports what the fold produced, and neither suppresses the other. Folding rewrites tokens and never kind, so the neighbouring one-restriction-per- kind rule cannot be created or destroyed by it. That was assumed by the split between the tiers and is now asserted. Cost, measured on this machine over a worst-case conformant term (8 restrictions, each 64 permitted and 64 prohibited tokens, none colliding, so the walk never returns early): ValidateLicenseTerm goes from 346us to 392us. The prohibited list is folded once into a set, so the added work is one fold per token; a pairwise walk would have folded four thousand times per restriction.

… Python The Go oracle gained the check; the two ports replay its corpus, so they carry it in the same position, with the same scan order and the same message bytes. Both already dispatch through their own canonicalRestrictionToken, so neither grows a second fold. The rule id joins the two hand-listed sets that decide which findings the entry corpus records as term rules rather than counting as structural. Both are edited here so the fix stands on its own; deriving them from the corpus is a separate change, made once every vector is passing. Python re-exports the constant from the package aggregator: the API-surface gate holds TypeScript-present-Python-absent at a hard zero, and the constant is public in both. The symbol map and the generated matrix take one row each — 131 symbols at parity, with the documented-divergence and exclusion counts unmoved, because this adds a face all three languages carry.

…tement The contract declared one disjointness rule and described one, so an implementor reading it built an Exchange that stores the term the SDK refuses. The rule is declared at the wire tier and compares the tokens as written; several tokens have more than one accepted spelling, so two spellings of one token clear it and become one token when the ingest tier folds them. That second evaluation existed nowhere in the prose. Both comments now say it. The rule's own comment says what it compares and why that is not the whole property, and names the ingest-tier id. The service comment, which is where a publisher reads what an Exchange will run, carries the reject in its ingest-tier list and states that a deployment which does not mount the wire tier still gets the second check. A behaviour stated nowhere in the contract is inferred from whatever implementation the reader has at hand, which makes an unstated verdict a defect in the contract rather than a gap in the docs — the reason this is a contract change and not a documentation one. The new guard holds the two together, the way the registration-schema and domain guards hold theirs. That the rule fires is read out of the committed vectors, as data, because this package is the tier below the SDKs and imports none of them. That the contract states it is read from the .proto SOURCE, because a comment does not survive into the descriptor and the comment is what an implementor reads. It matches clauses only these sentences can produce: the bare words would have passed on text that never states the rule, since "canonicalised" occurs in unrelated paragraphs and "disjoint" occurs in the wire rule's own id. A second test pins that the two rules keep two names and that the wire rule still exists, since the ingest-tier one reads different values and never replaced it. Only descriptor.binpb moves in gen/: the Go and TypeScript emitters carry no proto comments, and the descriptor is what the reference pages render from.

… say so Every surface described a two-tier model in which a token cannot be both permitted and prohibited was settled at the boundary. It is not: the boundary rule compares the tokens as written, and the property has to be asserted again over what the fold produced, because a registered alias and its canonical form are two spellings of one token. The licensing-terms page had not mentioned disjointness at all while describing canonicalisation two sections later, which is the pairing that hides the gap; it now states both checks and what separates them. The reference page carried the wire rule as though it were the whole rule. The ingestion, onboarding and content-ingestion pages describe the publisher pre-check, so they name the refusal a feed written against the registry's alias spellings will actually hit. The request-flows page already explained that the wire tier is something a deployment turns on, which makes it the right place to say that the ingest check is what a mount without it still gets. The design-history entry records the general rule rather than this instance: a rule comparing two token-valued fields is only correct on canonical values, so an axis that gains canonicalisation obliges a re-read of every rule over it — move the rule to the tier that sees canonical values, or move canonicalisation ahead of the tier that carries the rule. It also records why the second half has its own id, why it folds internally, and why neither half suppresses the other, including the concrete ground: the server binding defaults to validation off, so on a mount that never asks for the wire tier the ingest check is the only one a pushed term meets.

…them thrice Sorting an entry's violations into term rejects, cross-field rules and structural faults needed the same set of ingest-tier ids written down in four places: the emitter, the Go replay and the two ports. Adding one reject meant four edits, and the failure of missing one is quiet — a violation lands in the structural bucket and the entry assertions still pass. The three replays now read the set out of the corpus, because an ingest-tier reject IS an id the per-term list can carry as a violation. That is the trade the neighbouring cross-field set already makes, and for the same reason. The emitter cannot derive it — it writes the corpus the others read — so it keeps the one authored copy, beside the checks it classifies. Each derivation carries the vacuity guard the ids guard in conformance already uses: an empty set would move every term reject to structural and leave the assertions green, so both long-standing ids are required to be present, which pins that the column is still being read. The corpus is byte-identical. This changes how the replays classify, not what they assert.

…uages The refusal's message is wire-visible — an Exchange copies it into the rejection it answers a publisher with — so it is pinned byte for byte across the three SDKs, the committed corpus, and a byte-compare on the consuming side. It was not actually identical. Go formatted the token with %q and the two ports interpolated it raw, so the three answers agreed only on tokens that need no escaping. A token carrying a quote, a backslash or a control byte produced three different strings from one input. The corpus could not see it: every token in it is lowercase alphanumerics and hyphens, on which the two verbs agree. Go now spells the token the way the file's three other messages already spell it. The direction is deliberate. Aligning the ports to %q would mean reimplementing Go's escape syntax in TypeScript and Python byte-exactly, a new cross-language hazard for no gain, while aligning Go to the ports makes the fourth message behave like the other three: the neighbouring pricing-unit and quota-metric messages have always interpolated raw in all three languages, and their fields carry the same wire pattern, so this narrows a difference rather than widening an exposure. The corpus regenerates byte-identically. The two JSON ports also stop coercing an ill-typed token to the empty string. Their string accessors turn anything else into "", and the disjointness check is the one check in the file that cannot skip an empty token — two empty strings really are one token named on both sides, which is what the Go oracle answers — so a list carrying a number and a null was reported as a collision on the empty token. That names the wrong fault; the real one, "this element is not a string", is what the wire tier already reports beside it. An ill-typed element is now skipped, on the same division the file already applies to empty and vendor-namespaced tokens. Go needs nothing here: its token lists are []string. That behaviour cannot reach the shared corpus, because the Go oracle cannot construct a non-string token any more than it can construct an entry that is a string. It is pinned where the repo already pins that class, in the two port-local tests that mirror each other and say in their header why no vector can exist.

…'s answer is The corpus's per-term list holds one case per face; the per-entry list holds the composition an Exchange actually runs. A term that is disjoint only after folding was in the wrong one. Read as written, a bare alias is an unregistered token and earns a warning. Read after the fold it is a registered token and earns none. The per-term list records the term as authored, so it recorded a warning that no Exchange ever sends — which is not a difference a consumer can express by rewriting a token, because the warning does not change, it disappears. The per-entry list folds a copy first, exactly as the Exchange does, so the same case there records the answer a publisher is really given. Its neighbour already pins that asymmetry for membership; this now pins it for disjointness. What stays in the per-term list is the half that does not depend on where it is read: every alias-pair refusal, because the finding names the CANONICAL token and is therefore the same whether the caller folded first or not. That is the coverage the rule needs, and it is unmoved. The derived alias table now names every axis that CAN carry aliases rather than every axis that does today. GEOGRAPHY registers none, so it contributes nothing now and contributes a vector the day it registers one. Listing only the axes with aliases made the claim "an alias added to the proto brings its vector with it" false for the axis nobody was thinking about — which is the shape of the defect this whole rule exists to answer.

Four comments described the file as it was before the canonical-disjointness check existed. The new rule id was inserted between an existing Sphinx comment and the constant it described, so Python's public documentation said the REJECT constant "warns … the term is accepted" while the warn constant was left with none at all, and the new one carried a trailing docstring where every sibling uses the module's comment idiom. Both constants now carry their own description, in that idiom. The two port headers still enumerated the ingest tier's hard rejects as an unregistered unit or metric. They now name the third, matching the Go header and the contract. The per-term verdict was documented in all three languages as reading an already canonical term. It no longer does, and deliberately: the disjointness check folds what it compares. The generator's own note says which half of the list is which, because that is what a maintainer reads to know what may be added to it — a case that is accepted must stay canonical, since a bare alias read as written earns a warning the fold takes away, and such a case belongs in the entry list. The cost note justified itself with the per-list cap while naming, two paragraphs above, the mount that has no wire tier and therefore no cap. The work is linear in the tokens present; what bounds how many arrive is the cap where the wire tier runs, and the size of the request the server agreed to read where it does not. Citing only the cap claimed a bound the one deployment this check exists for does not have.

…ated The ingest tier's hard rejects are enumerated in several places, and adding one updated two of them. The contract's own all-or-nothing paragraph still described that tier as refusing an unregistered unit or metric, sixteen lines below the enumeration that had been corrected — and that paragraph is what a third-party implementor reads to know what a push costs. The design record and the feed-format page carried the same omission. All of them now name it. The guard added with the rule does not catch this class: it asserts that a clause is PRESENT, not that an enumeration is COMPLETE. The contract also said, and still says, that a token cannot be both permitted and prohibited on the same axis — while both rules that enforce it are scoped to a single restriction. Split the two lists across two restrictions of one kind and neither sees a collision; the one-per-kind rule refuses that shape instead. The sentence predates this work but sat beside a new one about a deployment without the wire tier, which together read as a promise neither rule keeps. The axis property is now stated as what it is: held by two rules together, one of which is a wire-tier rule. The new sentence is scoped to the reading it really adds. Whether the ingest tier should aggregate per kind, and so hold the axis property alone, is left open deliberately — deciding it would strengthen the rule beyond moving a check to the values it must read. Both changelogs said ten restriction tokens have more than one spelling. Ten aliases resolve to eight distinct tokens: `modify` and `commercial_entity` each answer to two.

main brought the Ed25519 signed-URL documentation sweep (18 commits, 29 files); this branch carries the post-fold disjointness reject. The two overlap in five files and genuinely conflicted in three. Resolution ledger: - mechanical: gen/descriptor.binpb -> REGENERATED from the merged proto, never content-merged (it is a binary generated artifact; buf 1.66.1). gen/go, gen/python/wire, gen/ts/wire regenerated with it, and conformance/corpus plus docs/sdk-parity-matrix.md re-derived, all byte-stable on a second run. - semantic: proto/CHANGELOG.md and website/.../reference/changelog.mdx -> UNION. Both sides prepended one entry under "## Unreleased" and main deleted nothing there, so the resolution takes main's file and re-inserts this branch's entry on top, matching the file's newest-first ordering. Both entries survive in both mirrors, which the repo requires of that pair. - auto-merged, verified not merely marker-free: proto/ramp/v1/ramp.proto keeps main's delivery-URL scheme (31 Ed25519 mentions, unchanged count) beside this branch's Restriction and CatalogService comments; publisher-onboarding.mdx differs from main's copy by exactly this branch's added clause. Verified after resolution: go build/vet/test, buf lint, doc-conformance (including the denylist patterns main added this sweep), parity-matrix drift, corpus determinism, canonical round-trip, sdk/ts 1496, sdk/python 1593, the CI-only API-surface gate, website 26 checks and a 79-page build.

legendko added 10 commits September 2, 2026 11:04

Navigation Menu

Uh oh!

Catalog push accepts a term that alias folding then makes invalid - #47

Catalog push accepts a term that alias folding then makes invalid#47
legendko wants to merge 10 commits into
mainfrom
fix/ramp-271-canonical-disjoint

legendko commented Sep 2, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Conversation

legendko commented Sep 2, 2026

Why

Decisions worth reading before the code

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant