Skip to content

fix(acp): request summarized thinking from Claude - #983

Open
Polarisyjr wants to merge 1 commit into
benchflow-ai:mainfrom
Polarisyjr:fix/claude-acp-thinking-summary
Open

fix(acp): request summarized thinking from Claude#983
Polarisyjr wants to merge 1 commit into
benchflow-ai:mainfrom
Polarisyjr:fix/claude-acp-thinking-summary

Conversation

@Polarisyjr

Copy link
Copy Markdown

Summary

Fixes #982.

Claude models that default thinking.display to "omitted" produce no usable thinking text in BenchFlow ACP trajectories.

This PR:

  • passes agent-specific _meta through ACPClient.session_new()
  • configures claude-agent-acp to request thinking.display="summarized"
  • keeps the configuration registry-driven instead of adding a Claude-specific runtime branch
  • ignores empty agent_thought_chunk and agent_message_chunk events so trajectory step counts only reflect actual content

Validation

Using the same task (lake-warming-attribution) before and after enabling summarized thinking:

Model Configuration Non-empty thought events Thinking chars
claude-sonnet-5 default 0 0
claude-sonnet-5 summarized 2 2126 (partial; stopped after confirming the result)
Haiku 4.5 default 9 3532
Haiku 4.5 summarized 9 3559

Added tests for:

  • forwarding _meta through session/new
  • omitting _meta when no session metadata is configured
  • Claude's summarized-thinking registry configuration
  • filtering empty thought/message chunks

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Claude ACP trajectories capture no thinking text on models that default thinking.display to "omitted"

1 participant