From a3dae4d7b8f12f1a71c53271e82ff875ce179693 Mon Sep 17 00:00:00 2001 From: Jaya Surya P <210020040@iitdh.ac.in> Date: Thu, 10 Sep 2026 16:53:59 +0530 Subject: [PATCH] [docs] document eval JSON paths --- src/pages/docs/observe/guides/setup-evals.mdx | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/src/pages/docs/observe/guides/setup-evals.mdx b/src/pages/docs/observe/guides/setup-evals.mdx index dcc9f9e6..c6e88cb9 100644 --- a/src/pages/docs/observe/guides/setup-evals.mdx +++ b/src/pages/docs/observe/guides/setup-evals.mdx @@ -38,6 +38,20 @@ The name autofills. For a built-in eval the instructions and output type are pre - **Error localization** (optional): tick it to pinpoint which part of the input caused a failure - **Variable mapping** (required): map each of the eval's inputs to your data. The choices are the attributes your project actually carries at the level you picked: span attribute keys for **Spans**, trace fields (`input`, `output`, `metadata`, and more) plus paths into each span for **Traces**, and session fields plus paths into their traces and spans for **Sessions**. Here, map the eval's `output` to the trace's `output` +### Map nested JSON attributes + +Use dot notation to map a nested object field. For arrays, use the numeric item position as a path segment, such as `.0` or `.1`; bracket notation such as `[0]` is not supported. + +A mapping can also start from a top-level span attribute whose value is a JSON object or array encoded as a string. The resolver parses that attribute, then continues walking the path. + +```json +{ + "attr_1": "{\"x\": {\"y\": [{\"z\": \"Your refund has been approved and will appear in your account within five business days.\"}]}}" +} +``` + +Map the eval variable to `attr_1.x.y.0.z` to pass `Your refund has been approved and will appear in your account within five business days.` to the evaluator. + Click **Add Evaluation** to attach it to the task. Toxicity eval configuration showing the read-only instructions, Pass/fail output, an error-localization checkbox, and a variable-mapping panel