fix(merge): strict item schema so OpenAI Structured Outputs accept the merge assessment request - #507
Merged
mortondev merged 2 commits intoSep 7, 2026
Conversation
…e request
`assessMergeCandidates` described its response items with
`z.record(z.string(), z.unknown())`. zod renders that with a `propertyNames`
keyword, which OpenAI's Structured Outputs reject:
400 Invalid schema for response_format 'structured_output':
In context=('properties', 'results', 'type', '0', 'items'),
'propertyNames' is not permitted.
The request is refused before the model runs, so every merge check fails and
no merge suggestion is ever created. `checkPostForMergeCandidates` then never
stamps `mergeCheckedAt`, so the sweep retries the same posts on every pass.
The loose item shape was deliberate — it let a single malformed item be
skipped instead of failing the whole batch. Under structured outputs that
tolerance is moot: the provider guarantees the shape. `results` keeps
`.catch([])`, so a present-but-wrong-shaped top level still degrades to
"no assessments" rather than failing the request, and the `typeof` guards in
the filter loop stay as they are.
Adds a regression test that asserts the schema handed to `chat()` emits no
`propertyNames`. It fails on the previous schema and passes on this one.
Fixes QuackbackIO#505
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
mortondev
approved these changes
Sep 7, 2026
mortondev
left a comment
Member
There was a problem hiding this comment.
Looks good — this is a real bug, not a speculative schema tweak.
z.record() really does emit propertyNames, OpenAI Structured Outputs reject that keyword, and a 400 means mergeCheckedAt never gets stamped so the sweep retries forever. The strict item schema matches the prompt, the regression test would fail on the old schema, and I applied the two nits (shorter comment, JSDoc spacing) on the fork.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #505.
What is broken
assessMergeCandidatesdescribes its response items withz.record(z.string(), z.unknown()). zod renders that as:{"type":"object","propertyNames":{"type":"string"},"additionalProperties":{}}OpenAI's Structured Outputs reject
propertyNames, so the request is refused before the model runs:Every merge check therefore fails against
api.openai.com. Worse,checkPostForMergeCandidatesonly stampsmergeCheckedAton success, so the sweep keeps retrying the same posts on every pass. Duplicate detection is permanently unavailable, and it is not model-specific — the schema is rejected, so switching chat models does not help.The change
Give the items the shape the system prompt already demands.
I want to be explicit that the loose shape was deliberate, and the comment says why: it let one malformed item be skipped rather than failing the whole batch. That reasoning does not survive structured outputs — the provider guarantees the shape, so there is no malformed item left to skip. Everything else is kept:
resultsstill has.catch([]), so a present-but-wrong-shaped top level degrades to "no assessments" instead of throwing.typeofguards in the filter loop stay untouched. They simply stop being load-bearing.Testing
Added one regression test to the existing suite. It asserts that the schema handed to
chat()emits nopropertyNames, taken from the actualmockChatcall rather than from an exported internal, so nothing new had to be made public.apps/webmerge-assessment suite: 13 passed (12 existing + 1 new)oxlintandprettier --checkclean on both filesVerified in production as well: with this change, a deliberately duplicated post is found by embedding similarity and confirmed by the model (confidence 0.95), and the merge suggestion is created. Running against
gpt-4.1-mini.Note
Unrelated to this PR, but worth flagging for anyone testing on OpenAI directly: the
max_tokensthis service sends is rejected by the gpt-5 family, which wantsmax_completion_tokens. That is a separate matter and not touched here.