feat: add summarization compactor

[sjrl]

Related Issues

• fixes https://github.com/deepset-ai/haystack-private/issues/512

Proposed Changes:

Added the experimental SummarizationCompactor, which progressively summarizes a conversation until it fits a target token budget. This preserves useful context from long-running Agents instead of dropping older messages outright.

The compactor reads the conversation as two regions. History runs from the end of the leading system messages up to the latest real user message; the current task runs from that user message to the end. It always summarizes history before the current task. Within each region, it summarizes original messages before combining existing summaries. When a historical turn contains both original messages and an existing summary, it summarizes the whole turn together to preserve chronological context. Each round uses the first applicable tier in this order:

1. historical_turns: Starting with the oldest, summarize as few turns that still contain original messages as needed to reach the target.
2. historical_summaries: When no original messages remain in history, combine as few of its oldest summaries as needed to reach the target.
3. current_task_steps: Summarize the fewest oldest steps needed to reach the target while preserving the min_keep_steps newest steps.
4. current_task_summaries: When no more steps can be summarized, combine as few of the current task's oldest summaries as needed to reach the target.

Each summary records the summarization strategy and the number of messages it replaced under the context_compaction key in its meta.

Compaction has a floor it cannot go below: the leading system messages, one combined historical summary, the latest user message, one combined current-task summary, and the min_keep_steps newest steps. Once the conversation is reduced to that state, compact returns None because there is nothing left that may be given up, even if the result is still above the target.

approximate_summary_tokens is the expected length of each summary. It is an estimate used to plan how much of the conversation to summarize, not a limit imposed on the model. A higher value summarizes more of the conversation per round and is more likely to bring the result under the target, at the cost of giving up more context. Configure any generation limit directly on the Chat Generator.

A generated summary is only applied when it reduces the measured conversation size; a response with no usable text is treated as a failure. By default, a failed summarization logs a warning and preserves any progress already made. Set raise_on_failure=True to propagate the error instead.

from haystack.components.agents import Agent
from haystack.components.generators.chat import OpenAIResponsesChatGenerator
from haystack.hooks.compaction import CompactionHook, SummarizationCompactor

agent_generator = OpenAIResponsesChatGenerator(model="gpt-5.4")
summary_generator = OpenAIResponsesChatGenerator(model="gpt-5.4-nano")

compaction_hook = CompactionHook(
    compactor=SummarizationCompactor(
        chat_generator=summary_generator,
        min_keep_steps=2,
        approximate_summary_tokens=1_024,
    ),
    context_window=400_000,
    compact_at=0.7,
    compact_to=0.4,
)

agent = Agent(
    chat_generator=agent_generator,
    hooks={"before_llm": [compaction_hook]},
)

How did you test it?

Added new tests.

Notes for the reviewer

Checklist

• I have read the contributors guidelines and the code of conduct.
• I have updated the related issue with new insights and changes.
• I have added unit tests and updated the docstrings.
• I've used one of the conventional commit types for my PR title: fix:, feat:, build:, chore:, ci:, docs:, style:, refactor:, perf:, test: and added ! in case the PR includes breaking changes.
• I have documented my code.
• I have added a release note file, following the contributors guidelines.
• I have run pre-commit hooks and fixed any issue.

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment

Project	Deployment	Actions	Updated (UTC)
haystack-docs	Ignored	Preview	Aug 19, 2026 12:54pm

[github-actions]

Coverage report

Click to see where and how coverage changed

File	Statements	Missing	Coverage	Coverage (new stmts)	Lines missing
haystack/hooks/compaction
__init__.py
summarization.py					89, 314-317, 475-479, 483-484, 488-492
haystack/token_counters
utils.py
Project Total

This report was generated by python-coverage-comment-action

[anakin87]

The implementation looks reasonable and quite clear to me

I left some (initial) minor comments

[anakin87]

Looks good!

Let's remember to document it in a future PR

[sjrl]

Looks good!

Let's remember to document it in a future PR

yup that's coming next!