Skip to content

docs: FicTrac columns in the MCAP bag — audit of the 25 columns + recommended record (LAB-303) - #263

Merged
mbreiser merged 2 commits into
mainfrom
docs/fictrac-columns-in-the-bag
Oct 19, 2026
Merged

mbreiser merged 2 commits into
mainfrom
docs/fictrac-columns-in-the-bag

Conversation

@mbreiser

@mbreiser mbreiser commented Oct 18, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Decision doc docs/development/fictrac-columns-in-the-bag.md for LAB-303 (ROS 2 bridge, MCAP as the record). Bags will be pushed to the GitHub data repo like the .jsonl.gz is today (Michael, 2026-10-18), so the bag has to stay near the JSONL's size. The doc:

  1. tables all 25 FicTrac output columns (meaning, units, which reader uses them, derivable or not), from FicTrac's doc/data_header.txt and src/Trackball.cpp;
  2. records the decision (Michael, 2026-10-18): keep 10 columns (the behavior row's 1, 15, 16, 17, 22 + the error score 5, the lab-frame delta rotation 6–8 and the sequence counter 23) in one per-frame /arena/frame record, float64, one MCAP message per frame, with /fictrac/frame and /fictrac/raw no longer recorded by default (live topics unchanged);
  3. measures that on the two 2026-10-14 bench bags (chunk-only index, zstd 19): 2.24× / 2.48× the .jsonl.gz, about 5 MB per 30 minute run at 50 Hz against 2.3 MB today (as recorded now 10.2× / 30.4×; all 25 columns 4.5× / 5.5×). Two levers were measured and declined for simplicity, available later: 1 s batching (1.94× / 2.14×) and int32 fixed-point at the JSONL's 5-decimal precision (≈ 1.5×);
  4. lists the code changes for the implementation PR (ArenaFrame.msg, bridge_node.py, _rows.py, mcap_convert.py + full level, recorder.py/mcap_index.py tuning, tests) without implementing them;
  5. ends with a proposed Linear comment for LAB-303 (text only, not applied).

Audit finding worth stating plainly: only the behavior row's 5 columns are read by anything today (bridge, kinematics.js, replay, dashboard, scripts, converter); the 25 are used only by mcap_convert --level full, whose own readers use cols 1/15–17/22 of it.

The doc references ros2-poc/ files that live on PR #254's branch (candidate 2026-10-20), not yet on main. No recorder code changes here; the sizing scripts are in the session notes and are listed for porting into ros2-poc/sizing/.

Tier

docs/skills-only.

Release notes

none

Test plan

  • Measurements reproduced from one script applied identically to every table row (both bags)
  • Derivability claims checked numerically on bag 1 (col 24 = Δcol 22 to 0.0; col 19 = hypot(Δx, Δy) to 1.6e-6; col 23 = col 1)
  • Michael decided: 10 columns, one record per frame, float64, no batching (2026-10-18)

🤖 Generated with Claude Code

mbreiser and others added 2 commits October 18, 2026 14:03
…ommended record (LAB-303)

Decision doc for the ROS 2 bridge's bag size now that bags go to the GitHub data repo:
which of FicTrac's 25 columns the bag must keep. Audits every reader (bridge, kinematics,
replay, dashboard, scripts, converter), recommends 10 columns in ONE per-frame /arena/frame
record (no raw FicTrac topics by default), measures the result on the two 2026-10-14 bench
bags with the plan's tuning (1.94x / 2.14x the jsonl.gz; today 10x / 30x), and lists the
code changes for the implementation PR. No recorder code changes here.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…er frame, float64, no batching

Michael 2026-10-18: adopt the recommendation, simpler is better, 2x accepted. Headline becomes the
per-record figure (2.24x / 2.48x the jsonl.gz, ~5 MB per 30 min run at 50 Hz); batching and
fixed-point stay documented as later knobs; the change list loses the batcher/unbatching items.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@mbreiser
mbreiser merged commit 27f6c95 into main Oct 19, 2026
4 checks passed
@mbreiser
mbreiser deleted the docs/fictrac-columns-in-the-bag branch October 19, 2026 02:50
mbreiser added a commit that referenced this pull request Oct 19, 2026
… derived-channel correction (LAB-303) (#265)

Follow-up to #263: the reproducible ros2-poc/sizing/columns.py (PR #264) measures the adopted
record at 2.41x / 2.68x the jsonl.gz (the first version quoted 2.24x / 2.48x from a scratch
script); the live turning_dps / forward_mms cost 16 B/frame of incompressible doubles, not
"about 1 %" (that came from a bench bag whose derived fields were the 2026-10-14 mixed-build
garbage), so the implementation records them as NaN. Links the implementation PR.

Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant