fix(train): analyze agent evolution once per commit without evaluator

[akushonkamen]

Description

CompressorV3.train_from_extracted_cases() loops over the N cases extracted from one session and runs analysis → gradient estimation → patch merge once per case. Without a rollout evaluator, the case never reaches the model: the default TrajectoryRolloutAnalyzer has no evaluator, so _evaluate_rollout() always returns None, extraction_messages are the session messages as-is, and case_name is only stamped onto the extracted trajectories afterwards. Every iteration therefore feeds byte-identical input through the whole pipeline — including thinking-enabled model calls — so a commit with N cases pays for the same whole-session evolution N times and merges N copies of the same patch (_merge_memory_diffs() does not deduplicate).

Fix (openviking/session/compressor_v3.py:1041-1162): when getattr(self.rollout_analyzer, "evaluator", None) is None, run the pipeline once for the first case, cache the RolloutAnalysis + RolloutTrainingResult, and reuse them for the remaining cases, linking each case to the shared outputs via _link_case_to_training_outputs(). Skill submission and the memory diff are computed once inside the single-analysis branch. When an evaluator is present, the per-case analysis is rubric-specific, so the per-case loop is preserved unchanged.

The old-behavior assertion test_train_from_extracted_multiple_case_memories_analyzes_bound_rollouts was updated to expect one whole-session analysis, and two tests were added (shared-pipeline behavior, evaluator-loop guard) — matching the approach proposed in the issue.

Scope note: at current main, openviking/session/policy_optimizer.py:186 already reads thinking=self.memory_type in {...} (experiences/trajectories still True), so the snippet quoted in the issue has drifted slightly. Making thinking follow configuration is the follow-up acknowledged in the issue and is intentionally not part of this fix.

Human Involvement

• A human participated in the implementation or review loop
• This PR was generated entirely by AI agents without human participation in the loop

Related Issue

Fixes #6006

Type of Change

• Bug fix (non-breaking change that fixes an issue)
• Test update

Changes Made

• Run the analyze → estimate → merge pipeline once per commit instead of once per case when the rollout analyzer has no evaluator; remaining cases link to the shared training result (_link_case_to_training_outputs()).
• Move skill submission and memory-diff computation into the single-analysis branch so they also run once (the merge does not deduplicate identical diffs).
• Preserve the existing per-case loop whenever an evaluator is configured.
• Update test_train_from_extracted_multiple_case_memories_analyzes_bound_rollouts and add test_train_from_extracted_multiple_cases_train_once_without_evaluator + test_train_from_extracted_multiple_cases_keeps_per_case_loop_with_evaluator.

Testing

• I have added tests that prove my fix is effective or that my feature works
• New and existing unit tests pass locally with my changes
• I have tested this on the following platforms:
  • Linux
  • macOS
  • Windows

Environment: uv sync --all-extras and OPENVIKING_CONFIG_FILE=/tmp/ov-test-6006.conf (minimal local config; this machine has no ~/.openviking — a pre-existing local-environment dependency, not a CI gap). All pytest runs with -q --no-cov.

Red = old source (base 6c11075) + this PR's test file:

uv run pytest tests/session/test_compressor_v3.py -q --no-cov
# 8 failed, 28 passed

Beyond the 6 pre-existing failures, both test_train_from_extracted_multiple_cases_train_once_without_evaluator (analyze ran twice for 2 cases) and the updated test_train_from_extracted_multiple_case_memories_analyzes_bound_rollouts (asserts one whole-session analysis) fail on the old code. Pristine baseline without the new tests: 6 failed / 28 passed.

Green = this PR:

uv run pytest tests/session/test_compressor_v3.py -q --no-cov
# 6 failed, 30 passed
uv run pytest tests/session/test_compressor_v3.py -q --no-cov \
  -k 'train_once_without_evaluator or keeps_per_case_loop_with_evaluator'
# 2 passed

The 6 failures are byte-identical to the pristine baseline (pre-existing, including the upstream test drift in test_train_from_extracted_cases_submits_streaming_rollout where FakeTrainer lacks batch_finalizer), so the "unit tests pass" checkbox stays unchecked, same policy as #6007.

Adjacency:

uv run pytest tests/session/train/test_trajectory_analyzer_component.py \
  tests/session/train/test_gradient_estimator_component.py -q --no-cov   # 8 passed
uv run pytest tests/session/test_session_commit.py -q --no-cov            # 20 passed
uv run pytest tests/session/train/ -q --no-cov
# 23 failed, 87 passed — FAILED list identical with and without the fix

Quality gates: ruff check passes on both touched files; ruff format --check reports only the pre-existing drift at lines 147/282/701 of compressor_v3.py (identical hunks on the untouched base file); mypy openviking/session/compressor_v3.py reports the same 7 pre-existing errors before and after — the diff is exactly 2 line-number shifts (1655→1676, 1742→1763), zero new findings.

Checklist

• My code follows the project's coding style
• I have performed a self-review of my code
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• My changes generate no new warnings
• Any dependent changes have been merged and published

No user-facing documentation covers this internal training-loop behavior; no doc changes were required. (Documentation checkbox left unchecked for that reason, not from omission.)

Additional Notes

• Test evidence above was re-run after rebasing onto main (6c11075, PR feat(cache): 增加顶层 service_name 配置， 删除 namespace/cache_key_prefix ，使用service_name 统一管理 #6011); that PR touches config/cache areas and does not overlap openviking/session/.
• Follow-up (out of scope here, acknowledged in the issue): let thinking follow configuration in policy_optimizer.py.