Skip to content

fix(builtin): decoders skip unknown fields and read absent fields as zero (1.0.0) - #238

Merged
facontidavide merged 2 commits into
mainfrom
fix/decoder-forward-compat
Nov 14, 2026
Merged

facontidavide merged 2 commits into
mainfrom
fix/decoder-forward-compat

Conversation

@facontidavide

@facontidavide facontidavide commented Nov 14, 2026 •

Copy link
Copy Markdown
Contributor

Why now

These three are decoder and schema decisions that 1.0.0 would freeze. Fixing them later changes how already-valid bytes are read.

1. Unknown fields inside a PointField

decodePointField read a varint before it knew the field number, then returned "unknown", and parseFields skipped a second field. Adding any varint field to PJ.PointField in a later release would have broken PointCloud, GridMap and VoxelGrid decoding on every 1.0.0 host.

parseFields now applies one rule to every codec:

Handler returns Meaning Loop does
true field consumed continue
false, nothing read unknown field or unexpected wire type skip it
false, something read known field is malformed fail the decode

Before, the third case skipped the bytes that followed and could decode as something else. I audited the handlers of all 20 decoders for a read that precedes the field check; decodePointField was the only one.

2. Absent fields are the proto3 zero value

A standard protobuf encoder omits a zero. Decoders that started from the C++ struct defaults read:

Sent Decoded before
PointCloud.is_dense = false true
PointCloud.height = 0 1
PointField.count = 0 1
CylinderPrimitive.top_scale = 0 (a cone) 1 (a cylinder)
annotation colour with a = 0 opaque

They now start from the zero values, and the empty PointCloud payload agrees with them. ImageAnnotations colours use the shared colour decoder. A NaN colour channel decodes as 0 instead of reaching an undefined float-to-integer cast.

Bytes written by the SDK's own serializers are unaffected: they write every one of these fields.

Also in this class, found by a Codex review of this PR:

  • An absent annotation thickness, circle diameter or font_size decoded as 2, 1 and 14. They now decode as 0. A thickness of 0 is a real value (a filled shape with no outline) that a standard encoder omits. A consumer that wants a default for 0 applies it when drawing.
  • deserializeBuiltinObject(kMeshMap, ...) accepted the empty payload as a default mesh with position_type FLOAT32, while any other payload that omits the field is rejected. The empty payload is now rejected too.

One exception remains, on purpose: a colour (or Mesh3D.scale) that is absent as a whole submessage keeps its documented default.

3. Log.category moves from field 5 to 7

Fields 5 and 6 are file and line in foxglove.Log, so a Foxglove-encoded log decoded its file path as the category. Both are now reserved. Only 1.0.0-dev.1 shipped field 5; a log encoded by a dev.1 build loses its category.

Verification

  • The first commit's nine tests were run against the unchanged decoders first: eight failed there. The ninth (NaN channel) passed, because the old behaviour was undefined rather than visibly wrong on this compiler.
  • The second commit's tests were not separately run against the old code.
  • With the fixes: ./build.sh --debug && ./test.sh gives 107/107 under ASAN. Pre-commit is clean.
  • A Codex review read all 43 parseFields call sites and the hand-written loops and found no handler the new rule misclassifies. Its two findings are the second commit.
  • Not run: ./test_sdk_install.sh (no packaging change).

docs/builtin_type.md now states the two decode rules.

🤖 Generated with Claude Code

facontidavide and others added 2 commits November 14, 2026 09:31
…zero (1.0.0)

Three wire decisions that a 1.0.0 host would otherwise have frozen.

Unknown fields. decodePointField read a varint before it knew the field,
then reported "unknown", and parseFields skipped a second field. A field
added to PointField by a later SDK would have broken PointCloud, GridMap
and VoxelGrid decoding on every 1.0.0 host. parseFields now has one rule:
false with nothing read means unknown and is skipped; false after reading
means a known field is malformed and the decode fails. decodePointField
checks the field number before it reads.

Absent fields. A standard proto3 encoder omits a zero, so an absent field
is zero. Decoders that started from the C++ struct defaults turned
is_dense = false into true, height 0 into 1, PointField.count 0 into 1,
a cone into a cylinder, and a transparent annotation colour into an opaque
one. They now start from the proto3 zero values; the empty PointCloud
payload agrees. Bytes written by the SDK's own serializers are unaffected.
A NaN colour channel decodes as 0 instead of reaching an undefined cast.

Log.category moves from field 5 to 7, with 5 and 6 reserved: they are file
and line in foxglove.Log, so a Foxglove log put its file path in category.
Only 1.0.0-dev.1 shipped field 5.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…o rule

Review findings on the previous commit.

An absent annotation thickness, circle diameter or font size decoded as 2,
1 and 14. They are ordinary proto3 scalars, and a thickness of 0 is a real
value (a filled shape with no outline) that a standard encoder omits. They
now decode as 0; colours still keep their default while the whole colour
message is absent.

deserializeBuiltinObject(kMeshMap) returned a default-constructed MeshMap
(position_type FLOAT32) for the empty payload, while a payload that carries
any other field and omits position_type is rejected. The empty payload is
now rejected the same way.

Tests: both of the above, a cone with only top_scale written, and a
malformed PointField followed by a well-formed field.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
@facontidavide
facontidavide force-pushed the fix/decoder-forward-compat branch from 25d4cb5 to ce88a64 Compare November 14, 2026 08:31
@facontidavide
facontidavide merged commit 71d368e into main Nov 14, 2026
4 checks passed
@facontidavide
facontidavide deleted the fix/decoder-forward-compat branch November 14, 2026 08:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant