Skip to content

fix: extract LangChain content-block text and apply model parameters after eval - #80

Open
andrewklatzke wants to merge 1 commit into
mainfrom
aklatzke/AIC-3351/fix-content-blocks
Open

fix: extract LangChain content-block text and apply model parameters after eval#80
andrewklatzke wants to merge 1 commit into
mainfrom
aklatzke/AIC-3351/fix-content-blocks

Conversation

@andrewklatzke

Copy link
Copy Markdown
Contributor

Summary

  • Extract visible text from LangChain AIMessage.content when it is a list of thinking/text blocks so callers no longer get an empty output while tokens are still billed.
  • Accept llm as a factory (config) => model (in addition to a pre-built instance) and spread flag model.parameters unchanged into default ChatAnthropic/ChatOpenAI constructors after evaluation.
  • Add langchain-thinking example (tool-free prompt) plus unit tests for mixed content blocks, factory vs instance, and default constructor kwargs.

Test plan

  • packages/langchain-messages and packages/langchain-agents unit tests
  • python main.py langchain-thinking launch-darkly-documentation-summarizer-messages-claude "Reason it out yourself without any tools: what is 17 times 23?" — non-empty response
  • Same scenario with doesnt-exist — exit 1

Made with Cursor

…after eval

LangChain AIMessage.content can be a list of thinking/text blocks; keep the visible text for callers. Accept llm as a factory so ChatAnthropic/ChatOpenAI can be built with flag model.parameters after evaluation.

Co-authored-by: Cursor <cursoragent@cursor.com>

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 7beaa6f. Configure here.

model_name=str(model.get("name") or "claude-sonnet-4-5"),
thinking={"type": "enabled", "budget_tokens": _THINKING_BUDGET_TOKENS},
max_tokens_to_sample=_MAX_TOKENS,
)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Example factory kwargs collide with parameters

Medium Severity

The thinking example spreads flag model.parameters into ChatAnthropic and then passes overlapping keywords (timeout, stop, model_name, thinking, max_tokens_to_sample). A second value for any of those keys is a TypeError, and max_tokens aliases max_tokens_to_sample, so a typical Claude flag never constructs the model.

Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit 7beaa6f. Configure here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant