Skip to content

fix(deps): depend on oss-harmony, the harmony build vLLM uses - #204

Closed
willccbb wants to merge 1 commit into
mainfrom
harmony-dedupe
Closed

willccbb wants to merge 1 commit into
mainfrom
harmony-dedupe

Conversation

@willccbb

@willccbb willccbb commented Oct 26, 2026 •

Copy link
Copy Markdown
Member

Problem

openai-harmony and oss-harmony both install the openai_harmony module. renderers required openai-harmony, and vLLM >= 0.32 requires oss-harmony (vLLM #55128). So any environment with both, prime-rl's included, held two packages writing the same files, and whichever installed last supplied the native code. In prime-rl's locked env the .so on disk is openai-harmony 0.0.8's, so vLLM runs 0.0.8 code too, even though it asks for the fork.

What the fork changes (0.0.8 → 0.0.11, Rust only; the Python layer is identical)

  • Commentary-channel headers render channel-first: <|channel|>commentary to=functions.x instead of to=functions.x<|channel|>commentary. This applies to tool results as well as tool calls.
  • A bare json content type gets a real <|constrain|> token.
  • The header parser strips duplicate <|channel|> markers. On 0.0.8 they end up in the recipient, e.g. functions.bash<|channel|>commentary.
  • The tiktoken vocab is embedded in the binary instead of downloaded at runtime.

Change

  • Depend on oss-harmony>=0.0.11.

  • Keep the gpt-oss renderer's tokens unchanged. Which header order gpt-oss handles best in history is contested:

    • The fork's author (Red Hat) reports that channel-first history reduces malformed headers.
    • vLLM's #55128 claims the opposite and rewrites prior assistant tool calls back to recipient-first (_use_legacy_tool_call_headers).
    • OpenAI's chat template puts the recipient first for both tool calls and tool results.
    • OpenAI's harmony guide shows tool calls channel-first and tool results recipient-first.

    So _render_message keeps the recipient-first header for tool calls and tool results, encoding header text the way harmony does (no special-token parsing).

  • The parity oracle is harmony itself, so it applies the same header order (harmony_recipient_first). That rewrite is a no-op on 0.0.8. tests/test_gpt_oss.py pins the header as a literal string.

  • Drop the openai-harmony pins from the example scripts and note the SGLang collision in examples/README.md. SGLang pins openai-harmony==0.0.4.

Verification

  • An independent review swept 3,618 render cases against the original renderer on openai-harmony 0.0.8. They covered odd and special-token tool names, unicode, odd arguments, three configs, generation prompts and the bridge, and compared token_ids, message_indices, sampled_mask and is_content. All are identical on both builds.
  • -k "gpt-oss or gpt_oss or harmony": 230 passed, 3 skipped on oss-harmony and on openai-harmony 0.0.8.
  • The full suite was not run locally on the final commit; CI covers it.

Not in scope

renderers now produces the same tokens on either harmony build, so oss-harmony could move behind an extra to keep plain renderers installable next to SGLang. That's a call about SGLang support, and this PR leaves it alone.

🤖 Generated with Claude Code


Note

Medium Risk
Changes core gpt-oss tokenization for tool turns and the harmony dependency, which affects vLLM co-install and training/serving parity; behavior is guarded by parity tests and a dedicated header test.

Overview
Switches the package dependency from openai-harmony to oss-harmony>=0.0.11 (vLLM’s fork) and refreshes uv.lock, with comments explaining the shared openai_harmony module clash with SGLang’s pinned openai-harmony.

GptOssRenderer adds _render_message so commentary tool-call and tool-result headers stay recipient-first (to=… before <|channel|>) even though oss-harmony emits channel-first headers. render() and bridge_to_next_turn route per-message rendering through that helper so output tokens stay aligned with the prior openai-harmony contract.

Tests gain harmony_recipient_first on the harmony parity oracle and tests/test_gpt_oss.py to pin header ordering. Example scripts drop explicit openai-harmony PEP 723 pins; examples/README.md documents the SGLang harmony collision.

Reviewed by Cursor Bugbot for commit 5dc6315. Bugbot is set up for automated code reviews on this repo. Configure here.

openai-harmony and oss-harmony both install the openai_harmony module.
vLLM >= 0.32 requires oss-harmony while renderers required
openai-harmony, so environments with both held two packages writing the
same files, and whichever installed last supplied the native code.

oss-harmony renders commentary headers channel-first
(<|channel|>commentary to=functions.x). Which order gpt-oss handles best
in history is contested, so the renderer keeps the recipient-first
header it has always produced for tool calls and tool results. Rendered
tokens are unchanged, and identical on either harmony build.

The parity oracle is harmony itself, so it applies the same header
order; tests/test_gpt_oss.py pins it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@macroscopeapp

macroscopeapp Bot commented Oct 26, 2026

Copy link
Copy Markdown

Approvability

Verdict: Skipped

Macroscope could not complete its approvability analysis for this PR. Approvability analysis did not return a structured decision, so this PR cannot be approved automatically.

You can add or adjust custom eligibility rules. Learn more.

@willccbb willccbb closed this Oct 27, 2026
@willccbb
willccbb deleted the harmony-dedupe branch October 27, 2026 02:23
@willccbb
willccbb restored the harmony-dedupe branch October 27, 2026 03:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant