Skip to content

Carry each answer's tool calls into follow-up questions - #184

Draft
duaneatat wants to merge 1 commit into
mainfrom
db/chat-follow-up-context
Draft

duaneatat wants to merge 1 commit into
mainfrom
db/chat-follow-up-context

Conversation

@duaneatat

Copy link
Copy Markdown
Collaborator

Follow-ups had only the earlier answer's text, so the model rediscovered the dataset and places each time and couldn't change a chart it had drawn. The follow-up eval case was Haiku's weakest (82%).

  • The browser keeps each tool call's arguments and sends the successful ones back with the history
  • The server replays them as tool calls with a placeholder result; rows are never kept or re-sent
  • Only the agent's own tool names are accepted, with a cap on count and argument size
  • Chats saved before this still work and fall back to the chart-title note
  • The eval runner sends the same history, and saved runs now record each step's arguments
  • Checked in the app with the demo model: the follow-up request carries the earlier get_timeseries and render_chart calls
  • Not measured with a live eval run

History was text only, so a follow-up had to rediscover the dataset,
places, and filters, and could not change a chart it had drawn. The
browser now keeps each tool call's arguments and sends the ones that
worked back with the history; the server replays them as tool calls
with a placeholder result. Only the agent's own tool names are
accepted, within a size bound, and rows are never kept.
@duaneatat
duaneatat marked this pull request as draft November 12, 2026 22:39

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant