Skip to content

fix(bt_json): serialize nested values in logged dataclasses - #912

Merged
Abhijeet Prasad (AbhiPrasad) merged 5 commits into
braintrustdata:mainfrom
linjiaye929:codex/dataclass-nested-attachments
Oct 14, 2026
Merged

Abhijeet Prasad (AbhiPrasad) merged 5 commits into
braintrustdata:mainfrom
linjiaye929:codex/dataclass-nested-attachments

Conversation

@linjiaye929

@linjiaye929 Gareth.Y (linjiaye929) commented Oct 12, 2026 •

Copy link
Copy Markdown
Contributor

Summary

When you log a dataclass, Braintrust now copies its fields the same way it copies everything else you log. Attachments inside a dataclass's lists or dicts get uploaded, and nested objects come out as real JSON instead of Python's text form of the object.

@dataclass
class Result:
    files: list[Attachment]

span.log(output=Result(files=[Attachment(data=pdf_bytes, filename="report.pdf", content_type="application/pdf")]))
Before After
Logged output {"files": ["<braintrust.logger.Attachment object at 0x7f…>"]} {"files": [{"type": "braintrust_attachment", "key": "…", "filename": "report.pdf", …}]}
report.pdf uploaded? ❌ No, silently dropped ✅ Yes

Background: how logged values are sanitized

Before an event is sent, span.log(...) makes a safe copy of everything you pass with bt_safe_deep_copy. The copy walks your data and:

  • copies plain JSON values (str, int, dict, list, …) as they are
  • keeps Attachment objects so the background logger can upload them and replace them with a reference
  • replaces SDK objects (Span, Experiment, Dataset, Logger) with short placeholders like "<span>"
  • turns pydantic models into dicts, and NaN / Infinity into strings
  • replaces cycles with "<circular reference>" and stops at a maximum depth

The bug

A dataclass got a different, shallower treatment. Each field was converted once, and any list or dict inside a field went down a fallback path that turns objects into text. Attachments, pydantic models and nested dataclasses inside those containers became strings like "<…Attachment object at 0x…>" or "P(value=1)".

flowchart LR
    A["span.log(output=Result(...))"] --> B[bt_safe_deep_copy]
    B --> C{"dataclass?"}
    C -- "before" --> D["convert each field once"]
    D --> E["list / dict field: fallback converts objects to text"]
    E --> F["Attachment becomes a string, file never uploaded"]
    C -- "after" --> G["check special types first: Span, Experiment, Dataset, Logger, attachments"]
    G --> H["copy each field back through the same walker"]
    H --> I["Attachment kept, uploaded at flush, logged as a reference"]
Loading

What changes for users

Only dataclass values change. Plain dicts, lists, pydantic models and other values log exactly as before.

You log a dataclass with… Before After
an attachment in a list/dict field "<braintrust.logger.Attachment object at 0x…>", not uploaded attachment reference, uploaded
an attachment directly on a field attachment reference attachment reference (no change)
a pydantic model in a list field "x=1" {"x": 1}
a nested dataclass in a list field "P(value=1)" {"value": 1}
a Span / Experiment / Dataset / Logger in a list field "<braintrust.logger._NoopSpan object at 0x…>" "<span>" (etc.)
a set field "{1, 2}" [1, 2]
NaN in a list field whole object becomes "<non-sanitizable: P>" ["NaN"]
one unreadable or unset field whole object becomes "<non-sanitizable: Bad>" {"ok": 1, "unset": "<non-sanitizable: Bad.unset>"}
a reference back to itself "P(value=[...])" "<circular reference>"

Note

If you have scorers, dashboards or queries that match the old text strings (for example "P(value=1)"), they will now see structured JSON instead.

How it works

  1. One walker for everything. bt_safe_deep_copy sends a dataclass to _to_bt_safe along with the walker itself and the current depth. _to_bt_safe runs its usual checks in order (SDK placeholders, then attachments) and only then copies dataclass fields. Each field goes back through the walker, so nested containers, cycle detection and the depth limit work exactly as they do for dicts and lists.
  2. SDK objects stay opaque. A dataclass that subclasses Span, Experiment, Dataset or Logger still logs as its placeholder, so its internal fields never reach your logs. The same goes for attachment subclasses, which keep their special handling.
  3. Each field is copied on its own. If one field can't be read, only that field becomes "<non-sanitizable: Class.field>". The other fields, and any attachments in them, are still logged.
  4. Cleanups on the hot path. The lazy braintrust.logger import is cached, and the walker's four copies of the cycle-guard block are now one.

Performance

Measured in the same process against main, on Python 3.14.6:

Benchmark main This PR
deep copy, medium / large / circular payloads baseline within noise
deep copy of 1,000 small dataclasses 20.6 ms 6.8 ms (~3× faster)

Testing

Test Covers
test_dataclass_nested_attachments attachments directly on a field, in lists/tuples/dicts, in nested dataclasses and in lists of dataclasses
test_dataclass_sdk_object_subclasses_keep_placeholder dataclass subclasses of Span / Experiment / Dataset / Logger
test_dataclass_attachment_subclasses_keep_special_handling dataclass subclasses of BaseAttachment / ReadonlyAttachment
test_dataclass_cycles_and_depth_share_walker_state cycles, shared references and the depth limit through dataclasses
test_dataclass_unreadable_field_keeps_readable_fields per-field fallback
test_dataclass_attachments_reach_flush (logger) Attachment, ExternalAttachment and a dataclass attachment get uploaded at flush, and the original payload isn't changed
  • The new tests fail on main (9 of 10) and pass on this branch.
  • nox -s test_core: 990 passed, 69 skipped, 12 xfailed (Python 3.14.6). An earlier run on Python 3.12.14 with orjson 3.13.0 and Pydantic 2.14.0 also passed.
  • make lint (pre-commit hooks and pylint) passes.
  • Not tested locally: other Python versions, provider integration suites, and real attachment uploads.

Follow-up (not in this PR)

Output from pydantic model_dump() / .dict() isn't walked again. A Span nested inside a pydantic model stays a live object instead of becoming "<span>", and NaN there isn't converted. Nested attachments in pydantic models are already kept, so the bug fixed here doesn't apply.


AI assistance was used to implement and test this change.

🤖 Generated with Claude Code

@linjiaye929
Gareth.Y (linjiaye929) marked this pull request as ready for review October 12, 2026 10:32
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 12, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-14T21:00:09.160140Z f42392f New commits
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 2e27140463

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "Codex (@codex) review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "Codex (@codex) address that feedback".

Comment thread py/src/braintrust/bt_json.py Outdated
visited_discard(obj_id)

try:
if dataclasses.is_dataclass(v) and not isinstance(v, type):

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Preserve dataclass-based BaseAttachment instances

When a user implements the public BaseAttachment ABC as a dataclass, this branch matches before _to_bt_safe() can apply its BaseAttachment check, so bt_safe_deep_copy() converts the attachment into a plain field dictionary. _extract_attachments() only uploads values that remain BaseAttachment instances, so logging this otherwise-valid custom attachment silently skips its upload and reference replacement; the parent implementation preserved it by passing through _to_bt_safe(). Check the special attachment types before flattening dataclasses.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Confirmed and fixed in 30c6cbe. Dataclass BaseAttachment and ReadonlyAttachment subclasses now retain their existing special handling before generic dataclass traversal. Added 14 regression cases covering direct and nested serialization plus local logger extraction; all failed on the reviewed head and pass with the fix. The original dataclass regressions also remain green (28 focused tests total), and test_core passes 1004 tests. Logger coverage verifies extraction/scheduling and reference replacement without real uploads. AI-assisted follow-up.

Route dataclasses in bt_safe_deep_copy through _to_bt_safe so the Span,
Experiment, Dataset, Logger, and attachment checks run before fields are
copied. The walker passes its own field copier, so nested values keep
sharing its cycle and depth tracking instead of starting a new copy.

Copy dataclass fields one at a time so a single unreadable field no
longer drops the readable fields and their attachments. Share one cycle
guard across the walker's branches, cache the lazy braintrust.logger
import, and move the dataclass attachment fixture into test_helpers.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Fold the per-shape and per-type parametrizations into one case per
behavior: nested attachments, SDK object special handling, shared
cycle/depth tracking, and unreadable fields. Merge the two logger flush
tests and drop the input/output parametrization, which exercised the
same path. Remove the JSON-container test and the older dataclass
attachment test, both covered by the remaining tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Pass the deep-copy walker and the field depth to `_to_bt_safe` as plain
arguments so non-dataclass leaves no longer allocate a `functools.partial`.
Return the cached logger types as a NamedTuple and use it in
`_to_json_safe` instead of a second lazy import. Split the
string-dispatched special-type test into one for SDK object placeholders
and one for attachment subclasses.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@AbhiPrasad Abhijeet Prasad (AbhiPrasad) changed the title fix: preserve nested attachments in dataclass fields fix(bt_json): serialize nested values in logged dataclasses Oct 14, 2026
@AbhiPrasad

Copy link
Copy Markdown
Member

thanks for the PR Gareth.Y (@linjiaye929)

made some changes, and updated pr title and description

@AbhiPrasad
Abhijeet Prasad (AbhiPrasad) merged commit 5b0b410 into braintrustdata:main Oct 14, 2026
98 checks passed
@AbhiPrasad

Copy link
Copy Markdown
Member

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants