Skip to content

Run research helper calls on a fixed cheap model - #163

Merged
Programcode merged 1 commit into
mainfrom
cost/haiku-background-steps
Oct 22, 2026
Merged

Programcode merged 1 commit into
mainfrom
cost/haiku-background-steps

Conversation

@Programcode

@Programcode Programcode commented Oct 22, 2026 •

Copy link
Copy Markdown
Owner

Cost fix 2 of 7 from the cost analysis.

What changes

When a request does web research, the server makes two small model calls before the answer: one decides whether an Ask question needs a web search, and one writes follow-up search queries. Until now both ran on the model the user picked. A Pro user on Claude Opus 5.5 at high effort paid Opus prices for two short JSON replies on every researched request.

Both calls now run on Claude Haiku 5.5 at low effort. The model is one value, BACKGROUND_MODEL in supabase/functions/_shared/model-catalog.ts, with its effort next to it. The answer the user sees still comes from their chosen model. The helpers' token usage is still added to the parent request's ai_requests row, as before.

Affected features: Generate, Attack, Brainstorm, Factoids (follow-up queries) and Ask with web search or citations (search decision plus follow-up queries). Crossfire, Sparring and streaming Ask make no research calls and are unchanged. Jev routing and ranking calls already use their own fixed model.

Things to know

  • The usage row still carries the user's model in its model column while its token totals now mix Haiku and the user's model. The dollar cost comes from OpenRouter per call, so the total charge stays correct. Splitting it per model belongs to the cost-logging fix.
  • Haiku writes the follow-up queries, so search quality for Pro users could shift slightly. These are 2 or 3 short search strings, so I don't expect a visible difference.

Verification

  • npm run verify passed locally at this head: docs check, lint, typecheck, unit tests, 356 Deno edge tests, build, service worker test, and 30 desktop Chromium smoke tests. GitHub Actions is broken repo-wide, so this is a local pass only, not a CI pass.
  • Extended the existing Ask handler test to assert that the helper calls use BACKGROUND_MODEL and the answer uses the requested model. It fails against the old code and passes with this change.
  • Real-app check with verify-debatelab (fake provider): on an Opus account, every helper call in Generate, Attack, Brainstorm, Factoids and Ask (web search and citation mode, desktop and mobile) asked for Haiku 5.5, and every answer asked for Opus 5.5. Details in the PR comment.
  • Codex review of bd692f7: no findings.

Deploying needs the ai Edge Function redeployed; this merge alone doesn't change production behaviour.

🤖 Generated with Claude Code

The Ask search decision and the follow-up search queries used to run on
whatever model the user picked, so a Pro user on Opus at high effort paid
Opus prices for two tiny JSON calls per researched request. They now run
on BACKGROUND_MODEL (Claude Haiku 5.5, low effort), one value in
_shared/model-catalog.ts. The answer itself still uses the user's model,
and the helper usage is still added to the parent request's row.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Copilot AI balanced review requested due to automatic review settings October 22, 2026 10:56
@vercel

vercel Bot commented Oct 22, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
debate-navigator Ready Ready Preview Oct 22, 2026 11:00am UTC
debatelab-staging Ready Ready Preview Oct 22, 2026 11:00am UTC

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@coderabbitai

coderabbitai Bot commented Oct 22, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

ℹ️ Recent review info
⚙️ Run configuration
📥 Commits

Reviewing files that changed from the base of the PR and between 4f9e0a0 and bd692f7.

📒 Files selected for processing: 4
  • docs/architecture.md
  • supabase/functions/_shared/model-catalog.ts
  • supabase/functions/ai/handler-research.test.ts
  • supabase/functions/ai/index.ts

Included review availability: This review used your included allowance. Your plan provides up to 1 included review per hour; 0 remain after this review.

📝 Walkthrough

Research helper calls now use a fixed background model and reasoning effort. Answer generation continues to use the user-selected model. The Ask test checks the model used for each call, and the architecture documentation describes the change.

Changes: Background research model

Layer / File(s) Summary
Background model constants
supabase/functions/_shared/model-catalog.ts
Adds exported constants for the background model, anthropic/claude-haiku-5.5, and its "low" reasoning effort.
Research call wiring and validation
supabase/functions/ai/index.ts, supabase/functions/ai/handler-research.test.ts, docs/architecture.md
Research calls use the fixed background settings. The Ask test checks that two helper calls use the background model and the answer call uses the requested model. The architecture documentation describes the model assignments and usage accounting.

Estimated code review effort: 2 (Simple) | ~10 minutes

  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Note

🎁 Summarized by CodeRabbit Free

Your organization is on the Free plan. CodeRabbit will generate a high-level summary and a walkthrough for each pull request. For a comprehensive line-by-line review, please upgrade your subscription to CodeRabbit Essentials by visiting https://app.coderabbit.ai/settings/billing.

Comment @coderabbitai help to get the list of available commands.

@Programcode

Copy link
Copy Markdown
Owner Author

Real-app check with verify-debatelab (fake provider, local Supabase) at bd692f7. The demo user is on Claude Opus 5.5. The fake provider records the model each call asked for:

Recipe Helper calls Answer
attack-run research-queries → Haiku 5.5 Opus 5.5
brainstorm-run research-queries → Haiku 5.5 Opus 5.5
factoids-run research-queries → Haiku 5.5 Opus 5.5
gen-research research-queries → Haiku 5.5 Opus 5.5
ask-web (desktop, mobile) research-decision + research-queries → Haiku 5.5 Opus 5.5
ask-cited (desktop, mobile) research-decision + research-queries → Haiku 5.5 Opus 5.5
ask-stream none (streaming Ask does no research) Opus 5.5

All passed. ask-cited failed once on desktop during the first run: the click on the Options button timed out before any request went out. It passed when rerun alone on desktop and on mobile. The fake mode doesn't prove OpenRouter accepts Haiku with these parameters live. Haiku is already the Basic default and goes through the same callOpenRouter path, though.

Codex review (gpt-6.1-sol, high) of bd692f7: no findings.

@Programcode
Programcode merged commit e5850ee into main Oct 22, 2026
8 of 19 checks passed

This branch was successfully deployed

2 active deployments
Preview – debatelab-staging — bd692f7e Deployed Oct 22, 2026 by vercel[bot]
Preview – debate-navigator — bd692f7e Deployed Oct 22, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants