Skip to content

filter_ai_spans=True has been a no-op since 0.28.0 — on_end injects braintrust.context_json ahead of AISpanProcessor's prefix filter #651

Description

@jsloa071

filter_ai_spans=True has been a no-op since 0.28.0 — on_end injects braintrust.context_json ahead of AISpanProcessor's prefix filter

What happens

BraintrustSpanProcessor(filter_ai_spans=True) exports every span the tracer provider produces, not just AI spans. HTTP server spans, DB client spans, health-check probes — all of it reaches Braintrust.

Two changes are individually reasonable but interact:

  1. _with_span_origin_attributes unconditionally sets a braintrust.-prefixed attribute (braintrust.context_json) on every span.
  2. BraintrustSpanProcessor.on_end applies that injection before handing the span to AISpanProcessor, whose filter keeps any span with an attribute matching FILTER_PREFIXES — which includes "braintrust.".

So the injected key satisfies the filter on every span, and the filter can never say no.

Affected versions

version behaviour
<= 0.27.0 correct — filter works
0.28.0 broken
0.29.0 – 0.31.1 (latest) broken

Bisected to #570 "Add span origin provenance" (4e3911a36f, merged 2026-07-15), which shipped in 0.28.0 (tagged 2026-07-16 13:37Z). 0.27.0 (tagged 2026-07-14) is clean. That commit's diff adds both halves at once — the braintrust.-prefixed attribute and the on_end call site that injects it upstream of the filter.

Reproduction

Verified against braintrust==0.30.1; the code path is identical in 0.31.1.

"""Minimal repro: filter_ai_spans=True exports non-genAI spans on >=0.28."""
from importlib.metadata import version

from braintrust.otel import BraintrustSpanProcessor
from opentelemetry.sdk.trace import ReadableSpan
from opentelemetry.trace import SpanContext, TraceFlags


class Recorder:
    """Stands in for the inner BatchSpanProcessor; records what reaches export."""

    def __init__(self, _exporter=None):
        self.spans = []

    def on_start(self, span, parent_context=None): pass
    def on_end(self, span): self.spans.append(span.name)
    def shutdown(self): pass
    def force_flush(self, timeout_millis=30000): return True


def span(name, attributes):
    return ReadableSpan(
        name=name,
        context=SpanContext(trace_id=1, span_id=1, is_remote=False,
                            trace_flags=TraceFlags(1)),
        attributes=attributes,
    )


recorder = Recorder()
processor = BraintrustSpanProcessor(
    api_key="sk-dummy",
    parent="project_name:repro",
    filter_ai_spans=True,
    SpanProcessor=lambda _exporter: recorder,
)

processor.on_end(span("GET /health", {"http.route": "/health"}))
processor.on_end(span("SELECT users", {"db.system": "postgresql"}))
processor.on_end(span("chat", {"gen_ai.operation.name": "chat"}))

print(f"braintrust {version('braintrust')}")
print(f"exported: {recorder.spans}")
print(f"expected: ['chat']")

Actual:

braintrust 0.30.1
exported: ['GET /health', 'SELECT users', 'chat']
expected: ['chat']

(The SpanProcessor hook is only used to capture what reaches export without opening a network connection. The filtering behaviour is the same with the default BatchSpanProcessor.)

Root cause

BraintrustSpanProcessor.on_end — injects, then delegates to the filter:

def on_end(self, span):
    """Forward span end events to the inner processor."""
    self._exporter.initialize()
    self._processor.on_end(_with_span_origin_attributes(span, self._environment))

When filter_ai_spans=True, self._processor is the AISpanProcessor, so the span it filters has already been mutated.

_with_span_origin_attributes sets the key unconditionally:

attributes["braintrust.context_json"] = json.dumps(
    merge_span_origin_context(existing_context, "braintrust-python-otel", environment)
)

_should_keep_filtered_span matches it via FILTER_PREFIXES:

FILTER_PREFIXES = ("gen_ai.", "braintrust.", "llm.", "ai.", "traceloop.")

Supporting evidence that the injection point is the anomaly

_on_ending forwards the raw span, with no injection:

def _on_ending(self, span):
    """Forward pre-end hook when the wrapped processor supports it."""
    _forward_on_ending(self._processor, span)

So the pre-end hook filters correctly while on_end does not — the two paths disagree, which suggests the injection was added to one call site without accounting for the filter living behind it.

Impact

Anyone relying on the documented filter_ai_spans=True is silently exporting their full trace volume to Braintrust. It fails quietly: no error, no warning, just more spans and more billable volume than intended. We noticed only because non-AI span names started appearing in a project.

Suggested fix

Filter on the un-mutated span, then inject. Options, roughly in order of preference:

  1. Have AISpanProcessor receive the raw span and apply the origin injection inside its kept branch — e.g. compose as AISpanProcessor(_OriginInjectingProcessor(batch_processor)) so injection happens downstream of the filter decision.
  2. Keep the current composition but evaluate _should_keep_filtered_span against the pre-injection attributes.
  3. Exclude the SDK's own bookkeeping keys from qualifying a span (e.g. treat braintrust.context_json as not-a-signal). Narrower, but leaves the general shape — internal attribute added ahead of a filter that matches its prefix — able to recur.

A regression test that drives a non-AI span through a real BraintrustSpanProcessor(filter_ai_spans=True) and asserts nothing reaches the exporter would catch this class of bug; a test of _should_keep_filtered_span alone passes right through it, since the defect is in what the span looks like by the time the predicate sees it.

Workaround

Pass an explicit custom_filter that always returns a hard bool. _should_keep_filtered_span consults it before any prefix matching, and an explicit False short-circuits:

_MARKERS = frozenset({
    "gen_ai.conversation.id",
    "gen_ai.operation.name",
    "gen_ai.agent.name",
})


def only_genai(span) -> bool:
    attributes = getattr(span, "attributes", None) or {}
    return any(m in attributes for m in _MARKERS)


BraintrustSpanProcessor(filter_ai_spans=True, custom_filter=only_genai, parent=...)

Two notes for anyone copying this: filter_ai_spans=True must stay, since it's what installs the processor that consults custom_filter; and the predicate must return True/False and never None, because None falls through to the broken default.

Co-authored-by: Claude Opus 5 noreply@anthropic.com

Activity

  1. AbhiPrasad commented on Aug 4, 2026

    @AbhiPrasad
    Member

    hey Jodi Sloan (@jsloa071) thanks for opening an issue! we are on it

  2. AbhiPrasad commented on Aug 5, 2026

    @AbhiPrasad
    Member
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

No labels
No labels

Type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions