Skip to content

Resolve json tag names through encoding/json - #128

Merged
FumingPower3925 merged 2 commits into
mainfrom
fix-go127-tag-names
Aug 27, 2026
Merged

FumingPower3925 merged 2 commits into
mainfrom
fix-go127-tag-names

Conversation

@FumingPower3925

Copy link
Copy Markdown
Owner

Closes #126. main is red on Go 1.27, which joined the version matrix the day it went stable — the matrix derives from go.dev.

The break

Go 1.27 rebuilt encoding/json on top of encoding/json/v2 (type RawMessage = jsontext.Value) and changed which json: tag names it accepts. stdocs carried a copy of the old rule in isValidTagName, so the documented key and the wire key diverged:

tag wire key ≤1.26 wire key 1.27 stdocs documented
json:"🚀" Emoji (field name) 🚀 Emoji
json:"a\"b" X (field name) truncated to a X

The behaviour follows the toolchain, not the go directive, and GODEBUG=jsonv2=0 does not restore it. Since the supported floor is Go 1.24, the same source has to produce the right key on 1.24 through 1.27.

The fix

Stop mirroring the rule; ask encoding/json for the answer. parseJSONTag resolves a tag by marshalling a synthesized field that carries it, so the answer comes from the same encoding/json that will marshal the caller's values. Cached per tag.

The tag is probed under two differently named fields. A tag that really names a key yields that key twice; one encoding/json ignores yields each probe its own field name, and the disagreement signals the fallback. This matters: a single probe has to reserve a name to signal fallback, and a real tag can always spell it — a field tagged json:"StdocsWireProbe" would then be documented under its Go name while the wire carried StdocsWireProbe, and an embedded field would additionally be flattened, because the empty name also drives embedded and tagged in inspectField. Both shapes are regression-tested.

Deliberately scoped to name resolution. The BFS embedding/dominance logic from #70 still decides conflicts and flattening, and the tests confirm the composition is right: on 1.27 two tags that truncate to the same key collide, encoding/json drops both, and so does stdocs.

Tests

TestInvalidJSONTagNames pinned the rule that just changed, so it is replaced by TestTagNamesTrackEncodingJSON — a table that marshals each case with the real encoding/json and asserts the schema documents exactly the keys that appear, in both directions. It covers the old test's tags plus embedding, an outer tag shadowing a promoted one, a name that collides once truncated, and a tag spelling a probe's own field name.

Because it compares against a live marshal rather than a fixed expectation, it passes on both sides of the change instead of encoding either toolchain's answer.

Verification

Everything below was run under both GOTOOLCHAIN=go1.26.6 and go1.27.0:

  • gofmt/vet/build, full suite, -race, and the nested YAML module — green on both; golangci-lint 0 issues.
  • The fuzzer, whose corpus already crosses odd tags with odd field types: 3.6M execs clean on 1.27.
  • Negative control: reverting just the probe to the single-field form makes both sentinel cases fail on both toolchains, so the new tests genuinely catch the bug rather than passing vacuously.

An adversarial review pass over this change found the forgeable-sentinel bug above before it shipped; it also turned up two wrong-contract shapes that predate this work and reproduce on both toolchains, filed separately as #127.

Note

This unblocks #125 (the Scalar/Swagger bump), which is verified but sitting on red CI for this reason. It rebases onto this and releases as v0.9.3.

Go 1.27 rebuilt encoding/json on top of encoding/json/v2 and changed
which json: tag names it accepts. An emoji or a control character now
names a key where it used to be ignored, and a name holding a quote or
a backslash is truncated there rather than discarded. stdocs carried
its own copy of the old rule, so on Go 1.27 it documented Emoji for a
field the wire called 🚀 — a wrong contract that still built,
validated, and rendered.

The rule is no longer reimplemented, because it is not stable enough to
mirror: parseJSONTag now asks encoding/json which key a tag produces by
marshalling a synthesized field carrying it. The answer comes from the
same encoding/json that will marshal the caller's values, so the schema
tracks whichever toolchain compiled them, on 1.24 through 1.27 and
through whatever changes next. Answers are cached per tag.

The tag is probed under two differently named fields. A tag that really
names a key yields that key both times; one encoding/json ignores
yields each probe its own field name, and the disagreement is what
signals the fallback. A single probe would have had to reserve a name
to signal it, and a real tag can always spell that name — a field
tagged json:"StdocsWireProbe" would then have been read as a fallback
and documented under its Go name instead.

TestInvalidJSONTagNames pinned the rule that just changed, so it is
replaced by a table that compares the documented keys against a live
marshal instead of against a fixed expectation. It covers the tags it
used to, plus embedding, a name that collides once truncated, and a tag
spelling a probe's own field name.

ExampleWithParams reads its numbers as json.Number: Go 1.27 aliases
json.RawMessage to jsontext.Value, whose nil value prints as null
rather than nothing, which moved the example's output on that release
alone.

Closes #126
@FumingPower3925

Copy link
Copy Markdown
Owner Author

Re-triggering CI: the earlier runs were lost to the GitHub Actions outage (major outage 15:11Z–~16:0xZ), not to this change.

The Lint job installs a pinned golangci-lint and runs it against
whatever `stable` resolves to, which is now Go 1.27. golangci-lint
2.12.2 bundles a typechecker that predates it and rejects the 1.27
standard library outright:

    could not import math/rand/v2
    (rand.go:213:17: method must have no type parameters)

so the job failed with two typecheck errors in crypto/internal, none of
them in this repository. 2.13.0 added Go 1.27 support and carries the
x/tools releases that understand it; 2.13.1 is the current patch. Both
pins move together, the root module and the YAML round-trip module.

Verified with 2.13.1 against both toolchains, root and nested module:
clean on go1.26.6 and go1.27.0.
@FumingPower3925

Copy link
Copy Markdown
Owner Author

Added a second commit: the Lint job pinned golangci-lint 2.12.2, whose bundled typechecker predates Go 1.27 and rejects the 1.27 standard library outright (could not import math/rand/v2 ... method must have no type parameters) — two typecheck errors in crypto/internal, none in this repo. 2.13.0 added Go 1.27 support; both pins now move to 2.13.1.

That was the last red job: Test, Coverage, Spec validation and Go 1.27.0 all passed on the previous run.

(The earlier runs on this PR were lost to the GitHub Actions outage on the 26th, not to this change.)

@FumingPower3925
FumingPower3925 merged commit 94d9eff into main Aug 27, 2026
45 checks passed
FumingPower3925 added a commit that referenced this pull request Aug 27, 2026
Go 1.27 rebuilt encoding/json on top of encoding/json/v2 and changed
which json: tag names it accepts. An emoji or a control character now
names a key where it used to be ignored, and a name holding a quote or
a backslash is truncated there rather than discarded. stdocs carried
its own copy of the old rule, so on Go 1.27 it documented Emoji for a
field the wire called 🚀 — a wrong contract that still built,
validated, and rendered.

The rule is no longer reimplemented, because it is not stable enough to
mirror: parseJSONTag now asks encoding/json which key a tag produces by
marshalling a synthesized field carrying it. The answer comes from the
same encoding/json that will marshal the caller's values, so the schema
tracks whichever toolchain compiled them, on 1.24 through 1.27 and
through whatever changes next. Answers are cached per tag.

The tag is probed under two differently named fields. A tag that really
names a key yields that key both times; one encoding/json ignores
yields each probe its own field name, and the disagreement is what
signals the fallback. A single probe would have had to reserve a name
to signal it, and a real tag can always spell that name — a field
tagged json:"StdocsWireProbe" would then have been read as a fallback
and documented under its Go name instead.

TestInvalidJSONTagNames pinned the rule that just changed, so it is
replaced by a table that compares the documented keys against a live
marshal instead of against a fixed expectation. It covers the tags it
used to, plus embedding, a name that collides once truncated, and a tag
spelling a probe's own field name.

ExampleWithParams reads its numbers as json.Number: Go 1.27 aliases
json.RawMessage to jsontext.Value, whose nil value prints as null
rather than nothing, which moved the example's output on that release
alone.

Closes #126
@FumingPower3925
FumingPower3925 deleted the fix-go127-tag-names branch August 27, 2026 07:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Go 1.27 changes JSON tag-name handling and breaks the documented-vs-wire contract

1 participant