hermes: pressure-pass condensation of oversized user messages - #178
Merged
Conversation
hermes: make compaction trigger tunable via HERMES_COMPRESSION_THRESHOLD_PERCENT The compaction trigger was floored at MINIMUM_CONTEXT_LENGTH (64K) because model.max_tokens=65536 reserved half the 131072 window, collapsing the effective input budget (context_length - max_tokens) to 65536. Operators could not raise it: compression.threshold was undeclared (seeded nowhere). Add a HERMES_COMPRESSION_THRESHOLD_PERCENT env (agent.yaml passthrough) that the entrypoint seeds into the compression.threshold config key (the one agent_init.py actually reads, NOT compression.threshold_percent). Empty by default -> no-op for deployments that do not tune it. Operator box: paired with HERMES_MAX_TOKENS=40960 + threshold 0.80 in out/ordo.yaml site config, this moves compaction from ~64k to ~72k. Spec: docs/superpowers/specs/2026-08-14-hermes-compaction-threshold-design.md Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> @
A giant user paste (a 274KB exported transcript handed to Hermes as its task) deadlocked context compression on 2026-08-15: after the first compaction the whole active transcript sat inside the protected tail (protect_last_n covered every remaining message), the compressible middle was empty, and every demotion pass (#61932) spares user-authored rows — so preflight ended in no_progress on every attempt while the request could only grow. It crossed the model window (131,147 > 131,072 tokens), llamacpp fail-fasted it, and the agent burned 30-minute retry timeouts against a deterministic overflow (spilling 100-minute doomed prefills onto the CPU fallback whenever the GPU was leased away). Add a maintained patch against the pinned upstream (3c27eb62, v2026.8.3) extending the #61932 pressure pass with a final escalation: when tool and assistant demotion still leave the protected region over budget, condense the bulkiest NON-ACTIVE user messages to a head/tail excerpt with an explicit "[condensed under context pressure: N chars omitted]" marker. The active (last) user message is never touched; the full original stays in the session store. Deterministic, idempotent, string-content only. The patch carries upstream-style tests (5 new cases: condensation, active-message protection, idempotency, small-message exemption, multimodal skip); the full upstream compressor suite passes at the pin (113/113). Applied with `git apply --check` first so a future pin bump that breaks it fails the build loudly, per the llamacpp-patched pattern. Deployed live 2026-08-15 alongside an operator config change (protect_last_n 20 -> 8 in the daemon-owned config.yaml, edited with the gateway stopped) so post-compaction transcripts keep a real middle and the normal aux summarizer can fold old giant turns before the pressure pass is ever needed. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…builds The hermes and llamacpp-patched Dockerfiles COPY patch files into Linux build stages and run `git apply` on them. With autocrlf=true a Windows checkout rewrites the patches to CRLF, so their hunks no longer byte-match the LF upstream sources — the llamacpp pr24785-minimal.diff already sits CRLF in Windows working trees today and only applies by grace of git-apply tolerance. Pin the class LF both directions, same as the existing *.sh / *.yaml rules, and renormalize both patch files. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Incident (2026-08-15)
A 274KB exported transcript pasted as a Hermes task deadlocked context compression: after the first compaction the entire active transcript sat inside the protected tail, the compressible middle was empty, and every demotion pass (#61932 pressure pass included) spares user-authored rows. Preflight ended in
no_progresson every attempt while the request could only grow — it crossed the model window (131,147 > 131,072 tokens), llamacpp fail-fasted it, and the agent burned 30-minute retry timeouts against a deterministic overflow for hours (spilling 100-minute doomed prefills onto the CPU fallback whenever the GPU was leased away).Fix
Maintained patch against the pinned upstream (
3c27eb62, v2026.8.3), following theservices/llamacpp-patchedpattern (git apply --checkfirst — a future pin bump that breaks the patch fails the build loudly):[condensed under context pressure: N chars omitted]marker.Second commit pins
*.diff/*.patchto LF in.gitattributes— these files feedgit applyinside Linux build stages, and a Windowsautocrlfcheckout hands the builder CRLF hunks that no longer byte-match upstream sources (the existing llamacpp patch already sits CRLF in Windows working trees today).Deployed + validated live
Image rebuilt and gateway recreated 2026-08-15 (with the operator config change
protect_last_n: 20 → 8, edited with the gateway stopped per the daemon-owned config rule): gateway healthy, Discord reconnected, patch constants present in/opt/hermes-agent, no orphan agent-run containers.Candidate upstream contribution to NousResearch/hermes-agent.
🤖 Generated with Claude Code