Repository navigation
feat(langchain): emit AG-UI text and tool events - #163
Conversation
Handlers can yield the AG-UI text and tool subset. config().stream() still yields chunk then done. stream() is the opt-in entry. Co-authored-by: alexis <alexis@launchdarkly.com>
A handler that raises mid-call can still report the lifecycle without the tool result. Co-authored-by: alexis <alexis@launchdarkly.com>
Messages and agents map model chunks and the tool loop onto the shared subset. Argument text and results follow capture_content. Co-authored-by: alexis <alexis@launchdarkly.com>
|
bugbot run |
Later LangChain tool_call_chunks carry only an index. Remember the id from the first piece so those slices are not dropped. Co-authored-by: alexis <alexis@launchdarkly.com>
|
bugbot run |
Later chunks that omit the call id, and a tool_end that used to skip the complete arguments after a prefix, no longer publish truncated JSON. A failed turn also closes the calls it already opened. Co-authored-by: alexis <alexis@launchdarkly.com>
|
bugbot run |
jeffdupont
left a comment
There was a problem hiding this comment.
Reviewed at 462209b against base b7a9f3a (#164). langchain-messages 126 passed, langchain-agents 162 passed, client -k agui 16 passed. Findings 2–4 were reproduced with probes (a real create_react_agent with a fake model for 2; real langchain_core AIMessageChunks for 3 and 4).
Must fix
1. Spec §1.8 says handlers yield chunk; this PR stops doing that (langchain-messages/tests/test_handler.py:1175; monorepo TESTING.md §1.8 "Streaming", §3.15)
test_yields_chunk_events now expects TEXT_MESSAGE_CONTENT, so the spec test fails by design. config().stream() still yields chunks via StableProjector (client.py:255), but direct handler.stream() callers get a different shape. There's no AG-UI spec on any monorepo branch, and no js-ai-sdk counterpart.
Fix: land the AG-UI handler-event contract in §1.8 and open the JS counterpart before merging.
2. langchain-agents leaves tool calls open when a run fails (langchain-agents/.../handler.py:582-587)
The except Exception block closes spans and re-raises but never calls emitter.fail_open(). Reproduced with a raising tool: TOOL_CALL_START/ARGS/END for c1 and d1, then the raise, with no TOOL_CALL_RESULT. on_tool_error does append to finished_calls (spans.py:443-447), but that's drained only inside the astream loop. langchain-messages (:1064), #161 and #165 all call fail_open() here.
Fix: for rich in emitter.fail_open(): yield rich before raise at 587, plus a raising-tool test asserting each started call gets a failed TOOL_CALL_RESULT.
3. Streamed tool events and the tool loop disagree about which calls exist (langchain-messages/.../handler.py:834-884 vs :897-899, :961-966)
Event ids come from tool_call_chunks; the loop still uses accumulated_tool_calls = chunk.tool_calls from the last chunk that had any, and AIMessageChunk.tool_calls is parsed from that one chunk only (langchain_core/messages/ai.py:554-570, 1.6.4). With OpenAI-shaped chunks for two parallel calls, events correctly stream call_1/call_2, then the loop sees [{'name': '', 'id': None, 'args': {}}] and raises No handler registered for tool "", and fail_open reports both as failed though neither ran. With a split {"q + ":"x"}, events show {"q":"x"} but the tool gets {}. The loop bug predates this PR (reproduces on b7a9f3a), but these events can't be right on top of it. The tests hand-build FakeAIMessage with whole tool_calls (tests/test_handler.py:1184-1266), which hides it.
Fix: accumulate the LangChain way, gathered = chunk if gathered is None else gathered + chunk, use gathered.tool_calls, and take ids from gathered.tool_call_chunks by index. That replaces tool_chunk_ids/seen_call_ids (~15 lines) and makes event and loop ids the same by construction. Add one test with real AIMessageChunks.
Should fix
4. Argument "prefix repair" requires byte-identical compact JSON (client/.../agui.py:178-187)
{"q": "x (with a space) streamed and final {"q":"xy"} fails startswith, so the truncated JSON is published silently. Worse, when a later chunk carries whole tool_calls, LangChain synthesises a tool_call_chunk with spaced json.dumps (ai.py:518-528), streamed as a second delta: probe gives {"q": "x + {"q": "xy"}, invalid JSON. (Per-provider spacing unverified.)
Fix: once 3 is fixed this repair has no job; delete _arg_text and keep "emit full args only if nothing streamed".
5. The same TurnEmitter.tool_end edit is in #161 (agui.py:107,163,166-190). Whichever merges second conflicts; move it into #164 once if it survives 4.
6. Agents tool results come from new callback state when the graph already returns them (langchain-agents/.../spans.py:307-308,409-414,420-425,443-447,513-518; handler.py:533-543)
The tools step of agent.astream() already yields ToolMessages with tool_call_id, content and status="error" (langgraph prebuilt/tool_node.py:1011). The callback copy needs the OTel span handler (spans.py:549-550), and its id (tool_call_id or run_id) can differ from the message id (tc.id or tc.name, handler.py:525), which would open a second call.
Fix: handle msg.type == "tool" in the existing loop with emitter.tool_result(msg.tool_call_id, msg.content, failed=getattr(msg, "status", None) == "error"); delete the spans.py changes and the finished_calls loop (~35 lines).
7. No test covers agents tool events (langchain-agents/tests/test_handler.py has no TOOL_CALL). One success test (START/ARGS/END/RESULT per call, distinct parallel ids) and one raising-tool test; the latter would have caught 2.
Nice to have
- Cross-provider mapping differs: #161 closes text before
tool_start(noparentMessageId) while this PR and #165 keep it open. The missing-id fallback is the tool name here (tc.get("id") or tc["name"], messages:962,1006,1016, agents:525), so two parallel same-name id-less calls collapse into one (dedupe atagui.py:142,196); #161/#165 use"". Pick one rule in #164/the spec. - The
len(seen_call_ids) == 1fallback (handler.py:867-868) puts an id-less, index-less slice on the only call seen so far even if it belongs to a second call; 3's fix removes it. tests/test_handler.py:1285-1291: replace themessageIdconditional withany(e["type"] == "TEXT_MESSAGE_CONTENT" and e["delta"] == "visible answer" for e in events).
Verdict: fix 1, 2 and 3 first. ~−50 lines possible (3, 4, 6). Not checked: real provider streams through LangChain; all chunk shapes were hand-built AIMessageChunks.
The messages loop used the last chunk's tool_calls, which drops earlier parallel calls. Chunks are merged the way LangChain adds them, and a failed agent stream closes the tool calls it already opened. Co-authored-by: alexis <alexis@launchdarkly.com>
|
bugbot run |
Message ids are unique across runs, tool results always carry content, and a run cannot finish while a tool call is open. Argument text is completed only when it extends what was streamed. Disconnects close the handler, and callers can echo threadId and runId. Co-authored-by: alexis <alexis@launchdarkly.com>
Co-authored-by: alexis <alexis@launchdarkly.com>
The agent stream reports the tool result the graph already returned, instead of a second copy recorded by the span callback. Co-authored-by: alexis <alexis@launchdarkly.com>
|
Addressed the review. Tool-call accumulation and the failed-run close are in Must fix:
Should fix: |
|
bugbot run |
A streamed chunk can open a tool call that never appears in gathered.tool_calls. The success path then ended the turn without TOOL_CALL_END or TOOL_CALL_RESULT. Those calls are now closed as masked failures before the next turn, and a provider error closes the open text message after the span is failed. Co-authored-by: alexis <alexis@launchdarkly.com>
|
bugbot run |
2 similar comments
|
bugbot run |
|
bugbot run |
Handlers can yield the AG-UI text and tool subset. config().stream() still yields chunk then done. stream() is the opt-in entry. Co-authored-by: alexis <alexis@launchdarkly.com>
A handler that raises mid-call can still report the lifecycle without the tool result. Co-authored-by: alexis <alexis@launchdarkly.com>
Message ids are unique across runs, tool results always carry content, and a run cannot finish while a tool call is open. Argument text is completed only when it extends what was streamed. Disconnects close the handler, and callers can echo threadId and runId. Co-authored-by: alexis <alexis@launchdarkly.com>
contextlib.aclosing always awaits aclose. A handler may return a plain async iterator, and that call raised AttributeError after the loop had already succeeded. The tracker now closes the stream only when aclose is present. Co-authored-by: alexis <alexis@launchdarkly.com>
ae82b2c to
37c7387
Compare
The reader now sits on main, including configure(). Keep the LangChain AG-UI stream and that capture default so this branch merges into the reader. Co-authored-by: alexis <alexis@launchdarkly.com>
The reader and Claude streams are on main. Keep the LangChain argument-prefix test that main's copy of the reader tests does not have. Co-authored-by: alexis <alexis@launchdarkly.com>
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit e30bb95. Configure here.
A graph step can start a tool call and never send a tool message. The success path ended the text and yielded done with that call still open. It now closes those calls as masked failures before the run finishes. Co-authored-by: alexis <alexis@launchdarkly.com>
🤖 I have created a release *beep* *boop* --- <details><summary>launchdarkly-ai-server: 0.4.0</summary> ## [0.4.0](launchdarkly-ai-server-0.3.0...launchdarkly-ai-server-0.4.0) (2026-10-23) ### ⚠ BREAKING CHANGES * **evaluations:** keep the management API client internal ([#202](#202)) * **evaluations:** rename base uri to match spec ([#199](#199)) * **evaluations:** add test proving harness edge case matches spec ([#200](#200)) * **evaluations:** route handlers by provider and mode * **evaluations:** mark evaluations from code as experimental ([#157](#157)) ### Experimental * **evaluations:** add test proving harness edge case matches spec ([#200](#200)) ([fa7162e](fa7162e)) * **evaluations:** keep the management API client internal ([#202](#202)) ([17270ed](17270ed)) * **evaluations:** mark evaluations from code as experimental ([#157](#157)) ([1389765](1389765)) * **evaluations:** rename base uri to match spec ([#199](#199)) ([7d93f9c](7d93f9c)) * **evaluations:** route handlers by provider and mode ([f62591f](f62591f)) * **evaluations:** Set JSON as standard for output to pass to judge ([#203](#203)) ([ffc728b](ffc728b)) * **skills:** Correct the HTTP 403 advice now that server-side FDv2 is available to every account ([#178](#178)) ([b17e322](b17e322)) ### Features * **agui:** add a shared reader for handler streams ([#164](#164)) ([fc08dd5](fc08dd5)) * **AIC-3534:** add a TypeSafe Jev judge handler ([#149](#149)) ([1b41ea4](1b41ea4)) * **client:** add configure for SDK defaults ([#158](#158)) ([3736767](3736767)) * **langchain:** emit AG-UI text and tool events ([#163](#163)) ([ab5e9e1](ab5e9e1)) * **openai:** emit AG-UI text and tool events ([#161](#161)) ([2b1b2d4](2b1b2d4)) ### Bug Fixes * **AIC-3570:** re-resolve graph flags on every invoke and stream ([#187](#187)) ([8fdc4d4](8fdc4d4)) * **AIC-3571:** visit every child before its parent in reverse_traverse ([#189](#189)) ([87c73d9](87c73d9)) * **AIC-3572:** treat a variation without _ldMeta.enabled as disabled ([#184](#184)) ([7a0d260](7a0d260)) * **AIC-3574:** build the LaunchDarkly client off the event loop ([#185](#185)) ([bf03b6c](bf03b6c)) * **AIC-3576:** record deferred judge scores under the evaluated config ([#190](#190)) ([354d3f1](354d3f1)) * **AIC-3577:** send object tool results to the model as JSON ([#188](#188)) ([dead4bd](dead4bd)) * **AIC-3596:** name the [otel] extra in the missing-OTel warning ([#186](#186)) ([90ce7bf](90ce7bf)) * **client:** match a handler's provides_for after normalizing its mode ([f62591f](f62591f)) * give native graph spans their LaunchDarkly identity ([#121](#121)) ([e863b6d](e863b6d)) </details> <details><summary>launchdarkly-ai-python: 0.3.0</summary> ## [0.3.0](launchdarkly-ai-python-0.2.0...launchdarkly-ai-python-0.3.0) (2026-10-23) ### ⚠ BREAKING CHANGES * **evaluations:** mark evaluations from code as experimental ([#157](#157)) ### Experimental * **evaluations:** mark evaluations from code as experimental ([#157](#157)) ([1389765](1389765)) </details> <details><summary>launchdarkly-ai-claude-agents: 0.2.6</summary> ## [0.2.6](launchdarkly-ai-claude-agents-0.2.5...launchdarkly-ai-claude-agents-0.2.6) (2026-10-23) ### Features * **claude:** emit AG-UI text and tool events ([#165](#165)) ([3f3a0ce](3f3a0ce)) * **client:** add configure for SDK defaults ([#158](#158)) ([3736767](3736767)) ### Bug Fixes * **AIC-3571:** visit every child before its parent in reverse_traverse ([#189](#189)) ([87c73d9](87c73d9)) * **AIC-3577:** send object tool results to the model as JSON ([#188](#188)) ([dead4bd](dead4bd)) * **claude-agents:** fail the run when the CLI reports an API error as success ([#174](#174)) ([3d1a768](3d1a768)) * give native graph spans their LaunchDarkly identity ([#121](#121)) ([e863b6d](e863b6d)) </details> <details><summary>launchdarkly-ai-claude-messages: 0.2.5</summary> ## [0.2.5](launchdarkly-ai-claude-messages-0.2.4...launchdarkly-ai-claude-messages-0.2.5) (2026-10-23) ### Features * **claude:** emit AG-UI text and tool events ([#165](#165)) ([3f3a0ce](3f3a0ce)) * **client:** add configure for SDK defaults ([#158](#158)) ([3736767](3736767)) ### Bug Fixes * **AIC-3577:** send object tool results to the model as JSON ([#188](#188)) ([dead4bd](dead4bd)) </details> <details><summary>launchdarkly-ai-openai-agents: 0.2.6</summary> ## [0.2.6](launchdarkly-ai-openai-agents-0.2.5...launchdarkly-ai-openai-agents-0.2.6) (2026-10-23) ### Features * **client:** add configure for SDK defaults ([#158](#158)) ([3736767](3736767)) * **openai:** emit AG-UI text and tool events ([#161](#161)) ([2b1b2d4](2b1b2d4)) ### Bug Fixes * **AIC-3571:** visit every child before its parent in reverse_traverse ([#189](#189)) ([87c73d9](87c73d9)) * **AIC-3577:** send object tool results to the model as JSON ([#188](#188)) ([dead4bd](dead4bd)) * give native graph spans their LaunchDarkly identity ([#121](#121)) ([e863b6d](e863b6d)) </details> <details><summary>launchdarkly-ai-vercel-agents: 0.1.1</summary> ## [0.1.1](launchdarkly-ai-vercel-agents-0.1.0...launchdarkly-ai-vercel-agents-0.1.1) (2026-10-23) ### Features * **AIC-3408:** add Vercel AI SDK messages and agents adapters ([#103](#103)) ([64d679e](64d679e)) </details> <details><summary>launchdarkly-ai-litellm-agents: 0.1.1</summary> ## [0.1.1](launchdarkly-ai-litellm-agents-0.1.0...launchdarkly-ai-litellm-agents-0.1.1) (2026-10-23) ### Features * add LiteLLM messages and agents integrations ([#102](#102)) ([f2e9886](f2e9886)) </details> <details><summary>launchdarkly-ai-openai-messages: 0.2.5</summary> ## [0.2.5](launchdarkly-ai-openai-messages-0.2.4...launchdarkly-ai-openai-messages-0.2.5) (2026-10-23) ### Features * **client:** add configure for SDK defaults ([#158](#158)) ([3736767](3736767)) * **openai:** emit AG-UI text and tool events ([#161](#161)) ([2b1b2d4](2b1b2d4)) ### Bug Fixes * **AIC-3577:** send object tool results to the model as JSON ([#188](#188)) ([dead4bd](dead4bd)) </details> <details><summary>launchdarkly-ai-vercel-messages: 0.1.1</summary> ## [0.1.1](launchdarkly-ai-vercel-messages-0.1.0...launchdarkly-ai-vercel-messages-0.1.1) (2026-10-23) ### Features * **AIC-3408:** add Vercel AI SDK messages and agents adapters ([#103](#103)) ([64d679e](64d679e)) </details> <details><summary>launchdarkly-ai-litellm-messages: 0.1.1</summary> ## [0.1.1](launchdarkly-ai-litellm-messages-0.1.0...launchdarkly-ai-litellm-messages-0.1.1) (2026-10-23) ### Features * add LiteLLM messages and agents integrations ([#102](#102)) ([f2e9886](f2e9886)) </details> <details><summary>launchdarkly-ai-langchain-agents: 0.2.6</summary> ## [0.2.6](launchdarkly-ai-langchain-agents-0.2.5...launchdarkly-ai-langchain-agents-0.2.6) (2026-10-23) ### Features * **client:** add configure for SDK defaults ([#158](#158)) ([3736767](3736767)) * **langchain:** emit AG-UI text and tool events ([#163](#163)) ([ab5e9e1](ab5e9e1)) ### Bug Fixes * **AIC-3577:** send object tool results to the model as JSON ([#188](#188)) ([dead4bd](dead4bd)) * give native graph spans their LaunchDarkly identity ([#121](#121)) ([e863b6d](e863b6d)) </details> <details><summary>launchdarkly-ai-langchain-messages: 0.2.5</summary> ## [0.2.5](launchdarkly-ai-langchain-messages-0.2.4...launchdarkly-ai-langchain-messages-0.2.5) (2026-10-23) ### Features * **client:** add configure for SDK defaults ([#158](#158)) ([3736767](3736767)) * **langchain:** emit AG-UI text and tool events ([#163](#163)) ([ab5e9e1](ab5e9e1)) ### Bug Fixes * **AIC-3577:** send object tool results to the model as JSON ([#188](#188)) ([dead4bd](dead4bd)) </details> <details><summary>launchdarkly-ai-opencode-agents: 0.2.0</summary> ## [0.2.0](launchdarkly-ai-opencode-agents-0.1.0...launchdarkly-ai-opencode-agents-0.2.0) (2026-10-23) ### ⚠ BREAKING CHANGES * **opencode:** start and stop opencode serve in the handler ([#198](#198)) ### Features * **opencode-agents:** add an OpenCode handler ([#159](#159)) ([fc08e8c](fc08e8c)) * **opencode-agents:** emit the AG-UI text and tool subset ([#162](#162)) ([7677621](7677621)) * **opencode-agents:** open one OpenCode session per call ([#179](#179)) ([b225113](b225113)) * **opencode-agents:** render agent instruction templates ([#182](#182)) ([83175aa](83175aa)) * **opencode:** start and stop opencode serve in the handler ([#198](#198)) ([0748dcf](0748dcf)) ### Bug Fixes * **opencode-agents:** own the HTTP client ([#194](#194)) ([ed2d419](ed2d419)) * **opencode-agents:** skip judges and record structured history ([#177](#177)) ([6fadf1d](6fadf1d)) </details> <details><summary>launchdarkly-ai-typesafe: 0.1.1</summary> ## [0.1.1](launchdarkly-ai-typesafe-0.1.0...launchdarkly-ai-typesafe-0.1.1) (2026-10-23) ### Features * **AIC-3534:** add a TypeSafe Jev judge handler ([#149](#149)) ([1b41ea4](1b41ea4)) </details> --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please). Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

LangChain messages and agents map model chunks and the tool loop onto the shared AG-UI subset. Argument text and results follow
capture_content.config().stream()stayschunkthendone.Verified with the LangChain handler tests and
mypy.