Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 12 additions & 3 deletions docs/sdk-conformance.md
Original file line number Diff line number Diff line change
Expand Up @@ -132,11 +132,13 @@ eventName: "e2e.inside")` inside it
- plain span `broken-step` ended with an error whose message contains "boom"
(TypeError in TS, ValueError in Python)
- embedding span `embed-query` (model `text-embedding-4`, usage input 3)
- plain span `private-step` (input "should-not-appear", captureInput false)
- plain span `big-payload` (input = 70000 × "x")
- plain span `private-step` (input "should-not-appear", captureInput false, error
"PRIVATE_EXCEPTION" whose message and stack are omitted)
- plain span `big-payload` (input = 10000 × "x" + 30000 × "🤖")
- generation `priced-call` (custom-model-x/custom, usage 5/2, costUsd 0.5)
- generation `openrouter-call` (provider openrouter, no usage)
- generation `masked-step` (input "SECRET stuff")
- generation `masked-step` (input "SECRET stuff", error "SECRET exception" whose message
and stack are masked to "[redacted exception]")
- plain span `json-compat` (nested non-finite values, numeric key ordering, binary64 numbers,
Unicode and lone surrogates; serialized bytes also stored as a custom attribute)
- generation `advisor-call` (claude-sonnet-4-5/anthropic, executor usage 8/3 with 2 cache-read
Expand All @@ -159,6 +161,13 @@ eventName: "e2e.inside")` inside it

## Assertions (C1–C24)

Exception content follows the effective per-span capture policy in both SDKs. If either input
or output capture is off, C14 retains error status/type and the exception event but omits its
message and stack. With both enabled, C16 applies the content mask and truncation to
`exception.message` and `exception.stacktrace` before export. Mask failures omit the affected
field. The policy is per span, so an enclosing span with capture on records an error that propagates
to it. These rules do not sanitize raw OpenTelemetry events or arbitrary attributes.

| # | Assertion |
| --- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| C1 | One trace per session: framework `otel`, userId/sessionId/metadata, token rollup 33/12 with the chat span's nine modality totals, server-computed cost >= 0.5, trace name `support-agent` |
Expand Down
6 changes: 4 additions & 2 deletions packages/anthropic/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -66,8 +66,10 @@ Reconstructed output is bounded to 48 KiB and 1,000 items. The configured `maxAt
exceeds a capture bound sets `telemetry.dev.capture.truncated`. With a mask configured, incomplete
output is omitted because the mask cannot inspect the complete value. When output capture is
disabled, response IDs and models, finish reasons, usage, timing, and errors may still be recorded.
Capture flags do not gate stop sequences, caller-supplied metadata or raw attributes, or exception
messages and stack traces; redact those separately when needed. Request tools and `tool_choice` are part of the captured input, so `captureInput: false` removes them.
Capture flags do not gate stop sequences or caller-supplied metadata or raw attributes; redact those
separately when needed. Exception messages and stack traces are recorded only when both
`captureInput` and `captureOutput` are on, and pass through `mask`. Older `@telemetry-dev/sdk`
versions record them regardless of capture flags. Request tools and `tool_choice` are part of the captured input, so `captureInput: false` removes them.

## Native Claude SDK spans

Expand Down
11 changes: 7 additions & 4 deletions packages/cursor/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -141,16 +141,19 @@ task, because Cursor generates chat titles from the prompt. A subagent that
`subagentStop` reports with status `error` then records `SubagentError`
without its task text.

Span exception messages and stack traces, including the `error_message` of
`sessionEnd` and of a failed tool or `Task` call, are recorded only when both
settings are `true`. Older `@telemetry-dev/sdk` versions record them regardless of
the settings.

The two settings do not cover these values, which are always exported:

- The user email, workspace and file paths, and conversation, generation,
subagent, and tool call ids.
- Model, tool, and subagent names, and lifecycle metadata such as statuses,
durations, compaction counts, and edit counts.
- Error messages from Cursor and exception stack traces. The `error_message`
of `sessionEnd` goes to the `cursor.error.message` log attribute and to a
`SessionError` exception. The `error_message` of a failed tool or `Task`
call becomes the exception message of its span.
- The `error_message` of `sessionEnd` in the `cursor.error.message` log
attribute.

To export none of this, set `enabled` to `false`.

Expand Down
3 changes: 2 additions & 1 deletion packages/cursor/tests/cursor.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -835,7 +835,7 @@ describe("titles", () => {
captureInput: false,
turnName: "invoke_agent cursor",
subName: "invoke_agent explore",
message: "SubagentError",
message: undefined,
},
])(
"captureInput $captureInput governs prompt, task and title text in span names and errors",
Expand Down Expand Up @@ -878,6 +878,7 @@ describe("titles", () => {
expect(exception?.attributes?.["exception.message"]).toBe(message);

if (!captureInput) {
expect(exception?.attributes?.["exception.stacktrace"]).toBeUndefined();
const exported = JSON.stringify([
finished(spans).map((s) => [s.name, s.attributes, s.events]),
logs.getFinishedLogRecords().map((r) => [r.body, r.attributes]),
Expand Down
4 changes: 2 additions & 2 deletions packages/omp/src/extension.ts
Original file line number Diff line number Diff line change
Expand Up @@ -421,8 +421,8 @@ export function telemetryDevExtension(

if (!span) return;
toolSpans.delete(event.toolCallId);
// The SDK exports error messages without the capture check or mask
// that `output` gets, and a failed tool's result is its output.
// Older core versions do not gate or mask exception messages; a failed
// tool's result must still follow its output capture policy.
const policy = span.capturePolicy;
const message =
policy?.output && !policy.mask
Expand Down
12 changes: 8 additions & 4 deletions packages/omp/tests/omp.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -401,7 +401,7 @@ test("tool execution produces an execute_tool span with arguments and result", a

test.each([
{ name: "default capture", options: {}, message: "grep: .env:\nTOKEN=private-tool-output" },
{ name: "captureOutput: false", options: { captureOutput: false }, message: "tool bash failed" },
{ name: "captureOutput: false", options: { captureOutput: false }, message: undefined },
{
name: "a mask",
options: {
Expand Down Expand Up @@ -437,9 +437,13 @@ test.each([
expect(exception?.["exception.type"]).toBe("ToolExecutionError");
// Failed tool output is tool output: only captured, unmasked output may reach the exception.
expect(exception?.["exception.message"]).toBe(message);
const stack = String(exception?.["exception.stacktrace"]);
expect(stack.startsWith(`ToolExecutionError: ${message}\n`)).toBe(true);
expect(stack.includes("private-tool-output")).toBe(message.includes("private-tool-output"));
if (message === undefined) {
expect(exception?.["exception.stacktrace"]).toBeUndefined();
} else {
const stack = String(exception?.["exception.stacktrace"]);
expect(stack.startsWith(`ToolExecutionError: ${message}\n`)).toBe(true);
expect(stack.includes("private-tool-output")).toBe(message.includes("private-tool-output"));
}
},
);

Expand Down
2 changes: 1 addition & 1 deletion packages/openai/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -88,7 +88,7 @@ Chat completion streams are traced. Requests are sent unchanged by default, so t

Streamed chat output and finish reasons use separate budgets of up to 48 KiB and 1,000 items; the configured `maxAttributeLength` does not shrink these retention bounds; the core applies it to the exported attribute after the mask runs, with its `...[truncated]` marker, without setting `telemetry.dev.capture.truncated`. When present, `gen_ai.output.messages` contains bounded partial output. If output and/or finish-reason capture is incomplete, including when a choice ends without `finish_reason`, `telemetry.dev.capture.truncated` is `true`. With a mask configured, incomplete output is omitted because the mask cannot inspect the complete value. When output capture is disabled or omitted, response IDs and models, finish reasons, usage, timing, and errors may still be recorded.

Capture flags do not gate stop sequences, caller-supplied metadata or raw attributes, or exception messages and stack traces; redact those separately when needed.
Capture flags do not gate stop sequences or caller-supplied metadata or raw attributes; redact those separately when needed. Exception messages and stack traces are recorded only when both `captureInput` and `captureOutput` are on, and pass through `mask`. Older `@telemetry-dev/sdk` versions record them regardless of capture flags.

```ts
const openai = wrapOpenAI(new OpenAI(), { injectStreamUsage: true });
Expand Down
5 changes: 2 additions & 3 deletions packages/opencode/src/plugin.ts
Original file line number Diff line number Diff line change
Expand Up @@ -129,9 +129,8 @@ function failureError(name: string, message: string | undefined): Error {

/**
* Host error text can be tool output that the host also sends to the model,
* and the SDK exports exception messages without the capture check or mask
* that `output` gets, so the text is used only where the span captures output
* unmasked.
* and older core versions export exception messages without capture checks or
* masking. Keep withholding that text unless the span captures output unmasked.
*/
function hostError(
span: SpanHandle,
Expand Down
26 changes: 18 additions & 8 deletions packages/opencode/tests/opencode.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -212,7 +212,7 @@ test("tool before and after create an execute_tool span with arguments, result,

test.each([
{ name: "default capture", options: {}, message: "grep: .env:\nTOKEN=private-tool-output" },
{ name: "captureOutput: false", options: { captureOutput: false }, message: "tool bash failed" },
{ name: "captureOutput: false", options: { captureOutput: false }, message: undefined },
{
name: "a mask",
options: {
Expand Down Expand Up @@ -258,9 +258,13 @@ test.each([
expect(exception?.["exception.type"]).toBe("ToolExecutionError");
// The host error text is the tool result: only captured, unmasked output may reach the exception.
expect(exception?.["exception.message"]).toBe(message);
const stack = String(exception?.["exception.stacktrace"]);
expect(stack.startsWith(`ToolExecutionError: ${message}\n`)).toBe(true);
expect(stack.includes("private-tool-output")).toBe(message.includes("private-tool-output"));
if (message === undefined) {
expect(exception?.["exception.stacktrace"]).toBeUndefined();
} else {
const stack = String(exception?.["exception.stacktrace"]);
expect(stack.startsWith(`ToolExecutionError: ${message}\n`)).toBe(true);
expect(stack.includes("private-tool-output")).toBe(message.includes("private-tool-output"));
}
expect(span.startTime[0]).toBe(1_700_000_020);
expect(span.endTime[0]).toBe(1_700_000_021);
});
Expand Down Expand Up @@ -290,7 +294,7 @@ describe.each([
{
name: "captureOutput: false",
options: { captureOutput: false },
message: "session error",
message: undefined,
logMessage: undefined,
},
{
Expand Down Expand Up @@ -335,9 +339,15 @@ describe.each([
expect(span.status.code, name).toBe(SpanStatusCode.ERROR);
expect(span.attributes["error.type"], name).toBe("UnknownError");
expect(exception?.["exception.message"], name).toBe(message);
const stack = String(exception?.["exception.stacktrace"]);
expect(stack.startsWith(`UnknownError: ${message}\n`), name).toBe(true);
expect(stack.includes("private-attachment-listing"), name).toBe(message !== "session error");
if (message === undefined) {
expect(exception?.["exception.stacktrace"], name).toBeUndefined();
} else {
const stack = String(exception?.["exception.stacktrace"]);
expect(stack.startsWith(`UnknownError: ${message}\n`), name).toBe(true);
expect(stack.includes("private-attachment-listing"), name).toBe(
message !== "session error",
);
}
expect(span.attributes["gen_ai.response.finish_reasons"], name).toBeUndefined();
}

Expand Down
4 changes: 2 additions & 2 deletions packages/pi/src/extension.ts
Original file line number Diff line number Diff line change
Expand Up @@ -385,8 +385,8 @@ export function telemetryDevExtension(

if (!span) return;
toolSpans.delete(event.toolCallId);
// The SDK exports error messages without the capture check or mask
// that `output` gets, and a failed tool's result is its output.
// Older core versions do not gate or mask exception messages; a failed
// tool's result must still follow its output capture policy.
const policy = span.capturePolicy;
const message =
policy?.output && !policy.mask
Expand Down
12 changes: 8 additions & 4 deletions packages/pi/tests/pi.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -616,7 +616,7 @@ test("a successful tool execution produces an execute_tool span with arguments a

test.each([
{ name: "default capture", options: {}, message: "grep: .env:\nTOKEN=private-tool-output" },
{ name: "captureOutput: false", options: { captureOutput: false }, message: "tool bash failed" },
{ name: "captureOutput: false", options: { captureOutput: false }, message: undefined },
{
name: "a mask",
options: {
Expand Down Expand Up @@ -651,9 +651,13 @@ test.each([
expect(span.attributes["error.type"]).toBe("ToolExecutionError");
expect(exception?.["exception.type"]).toBe("ToolExecutionError");
expect(exception?.["exception.message"]).toBe(message);
const stack = String(exception?.["exception.stacktrace"]);
expect(stack.startsWith(`ToolExecutionError: ${message}\n`)).toBe(true);
expect(stack.includes("private-tool-output")).toBe(message.includes("private-tool-output"));
if (message === undefined) {
expect(exception?.["exception.stacktrace"]).toBeUndefined();
} else {
const stack = String(exception?.["exception.stacktrace"]);
expect(stack.startsWith(`ToolExecutionError: ${message}\n`)).toBe(true);
expect(stack.includes("private-tool-output")).toBe(message.includes("private-tool-output"));
}
},
);

Expand Down
21 changes: 16 additions & 5 deletions packages/sdk/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -78,16 +78,27 @@ other string attribute is cut at the cap without a marker, except that a cap of
`exportTimeoutMillis` 30000), `spanFilter`, `resourceAttributes`, `logLevel` (SDK diagnostics,
default `"warn"`), `fetch`, `waitUntil`, `onError`.

Core SDK error recording keeps failure status, error type, and an exception event. It includes
`exception.message` and `exception.stacktrace` only when both effective `captureInput` and
`captureOutput` are true, including per-span overrides: exceptions can echo either inputs or
outputs. The policy is per span: when an error propagates to an enclosing span whose capture is on,
that span records its message and stack. Included fields pass through `mask` with their exception
attribute keys, then serialization and truncation. A throwing mask drops the affected field and reports a generic error to `onError`
and SDK diagnostics without the hook's original message or cause. The original application error
is rethrown unchanged. These controls do not scrub error type names, metadata, identifiers, resource
attributes, or events written directly through raw OpenTelemetry APIs.

`captureInput: false` and `captureOutput: false` omit captured content. Provider integrations may
still record non-content metadata such as response IDs and models, finish reasons, usage, timing,
and errors. Capture flags do not gate stop sequences, tool descriptions and definitions (`gen_ai.tool.description`, `gen_ai.tool.definitions`), caller-supplied metadata or raw attributes,
or exception messages and stack traces; redact those separately when needed.
and error types. Capture flags do not gate stop sequences, tool descriptions and definitions
(`gen_ai.tool.description`, `gen_ai.tool.definitions`), caller-supplied metadata or raw attributes;
redact those separately when needed.

`mask` receives input and output content, system instructions, log messages, log attributes
other than numbers and booleans, and content that integrations pass to
`handle.serializeContent`. `ctx.key` names the attribute being written. It does not receive
span or propagated `metadata`, raw `attributes`, or exception messages and stack traces; redact
those separately.
`handle.serializeContent`, including captured exception messages and stack traces. `ctx.key` names
the attribute being written. It does not receive span or propagated `metadata` or raw `attributes`;
redact those separately. Log masking is independent of span capture flags.

Root spans remain independent by default. `init({ sessionMode: "process" })` groups otherwise
uncorrelated spans and logs using a fresh opaque UUID for each enabled SDK initialization.
Expand Down
5 changes: 3 additions & 2 deletions packages/sdk/src/capture.ts
Original file line number Diff line number Diff line change
Expand Up @@ -270,8 +270,9 @@ export function prepareCaptureValue(
if (cfg.mask) {
try {
masked = cfg.mask(value, { key });
} catch (error) {
reportError(cfg.onError, error);
} catch {
// Hook errors can contain the unmasked value or chain the application exception.
reportError(cfg.onError, new Error("mask hook raised; content dropped"));

return undefined;
}
Expand Down
8 changes: 4 additions & 4 deletions packages/sdk/src/config.ts
Original file line number Diff line number Diff line change
Expand Up @@ -14,8 +14,8 @@ export type { BatchOptions, ExportMode, LogLevel, SdkLogLevel } from "@telemetry

/**
* Redaction hook applied to captured content (input/output, log messages and attributes, and
* content passed to `SpanHandle.serializeContent`) before JSON stringification. Not applied to
* metadata or raw attributes.
* content passed to `SpanHandle.serializeContent`), including exception messages/stacks, before
* JSON stringification. Not applied to metadata or raw attributes.
*/
export type MaskFn<T = unknown> = (value: T, ctx: { key: string }) => T;

Expand All @@ -38,9 +38,9 @@ export interface TelemetryOptions {
exportMode?: ExportMode;
/** OTel sampling policy. Defaults to OTEL_TRACES_SAMPLER / OTEL_TRACES_SAMPLER_ARG. */
sampler?: Sampler;
/** Capture span inputs (args, prompts) by default. Default true. */
/** Capture span inputs (args, prompts). False also omits exception messages/stacks. Default true. */
captureInput?: boolean;
/** Capture span outputs (results, completions) by default. Default true. */
/** Capture span outputs (results, completions). False also omits exception messages/stacks. Default true. */
captureOutput?: boolean;
/** Redaction hook run on captured content before stringify/truncate. */
mask?: MaskFn;
Expand Down
Loading
Loading