[sanskarIN]

Problem

The optional HF synthetic baseline/LoRA benchmark compares output token totals and inference latency, but cannot tell whether a completion ended at EOS/FIM suffix or exhausted maxNewTokens. Token-limit exits may represent truncated code, so exact-match and timing summaries without this caveat can mislead.

Implementation

• Bump HF smoke protocol to lineloom-hf-synthetic-smoke-v7 and offline comparison protocol to lineloom-hf-synthetic-comparison-v6.
• Add pure, standard-library classify_generation_termination(): classify the actual generated ID sequence as stop-token if the last token is an allowed EOS/FIM suffix (including at the length cap), otherwise token-limit only if the output exactly reaches the cap. Reject unexplained early truncations, stop tokens followed by further generation, and malformed IDs.
• Aggregate source-free stopToken and tokenLimit exit counts overall and across all five languages; discard per-example token sequences and reasons after computing metrics.
• Strictly validate required count schemas, integer types, per-language totals and feasible generated-token bounds in compare_smoke.py, refusing incompatible or fabricated reports.
• Expose base vs adapter token-limit exit totals and deltas, plus TOKEN_LIMIT_EXITS_REQUIRE_REVIEW when applicable; a natural stop does not imply code correctness.
• Add offline regression tests and document the protocol and limits in model evaluation documentation.

Verification and limits

All benchmark tests use invented counts/IDs and checked-in original synthetic fixtures; there is no GPU run, no actual model evaluation, no checkpoint or training corpus accessed, no model weights bundled, and no extension or marketplace release. Existing opt-in checkpoints and no-source-output controls remain. Merge only after cross-platform CI, independent dependency audit and isolated VSIX candidate installation/upgrade pass.