Skip to content

fix(train): plan policy updates outside the policy-root tree lock - #6147

Open
akushonkamen wants to merge 1 commit into
volcengine:mainfrom
akushonkamen:fix/6095-plan-outside-lock
Open

akushonkamen wants to merge 1 commit into
volcengine:mainfrom
akushonkamen:fix/6095-plan-outside-lock

Conversation

@akushonkamen

Copy link
Copy Markdown
Contributor

Description

本 PR 修复训练引擎在策略根目录整树锁内执行远程 LLM 规划导致的并发阻塞(#6095)。

根因:openviking/session/train/engine.py 的 plan_and_apply 在 policy_set.lock() 整根树锁内依次执行 reload → policy_optimizer.plan(远程 LLM 调用,生产 trace 中为分钟级)→ apply。并发 session-commit 在 openviking/session/train/domain.py 的 pathlock_acquire_tree(timeout_secs=300.0)(原先硬编码)上等待,等满 300s 后终态失败。(实测基于 HEAD=6006fc34:锁调用在 domain.py:84,docstring 中 "timeout=None means wait indefinitely" 的承诺在 :72,实现从未支持 None;工单引用 :100 为行漂移。)

修法:把规划移到锁外——先无锁 reload 出快照并基于快照规划;拿锁后仅做 reload 校验快照仍然一致才 apply;不一致则基于新快照有界重规划(_PLAN_APPLY_MAX_REPLANS=2);重规划预算耗尽后,带 optimistic_apply_after_conflict 标记对最新快照守卫应用,由 updater 既有 per-item before_content 基线守卫把过期目标转为记录错误而非静默覆盖。同时 PolicySet.lock 增加 timeout_secs 参数(默认 300s 不变),并修正 docstring 与实现不符的 "None=无限等待" 承诺。

Human Involvement

  • A human participated in the implementation or review loop
  • This PR was generated entirely by AI agents without human participation in the loop

Related Issue

Fixes #6095

Type of Change

  • Bug fix (non-breaking change that fixes an issue)
  • New feature (non-breaking change that adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Documentation update
  • Refactoring (no functional changes)
  • Performance improvement
  • Test update

Changes Made

  • engine.py:plan_and_apply 改为"锁外快照规划 + 锁内校验后短临界区应用";快照冲突时基于新快照有界重规划(上限 _PLAN_APPLY_MAX_REPLANS=2);预算耗尽后带 optimistic_apply_after_conflict 标记守卫应用,过期目标由既有 per-item 基线守卫报错而非静默覆盖。
  • domain.py:PolicySet.lock 新增 timeout_secs 关键字参数(默认 300.0,行为不变),锁调用透传该参数;docstring 删除实现从未支持的 "timeout=None 无限等待" 承诺,改为说明临界区应保持短小、远程 LLM 规划须在锁外执行。
  • tests/session/train/test_fakes.py:新增 LockingAGFS(具备真实互斥、deadline 处理与持锁区间记录的树锁 fake);DummyVikingFS 补上 lock() 所需的 _async_agfs 属性。
  • tests/session/train/test_train_framework.py:新增 5 条锁回归测试——(1) 2s 规划不再在 1s 锁预算内阻塞并发写者;(2) 快照冲突触发有界重规划并对最新快照守卫应用;(3) 无竞争快路径仅规划一次并释放锁;(4) 锁超时参数透传、出错时释放租约。

Testing

  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes
  • I have tested this on the following platforms:
    • Linux
    • macOS
    • Windows

本地验证(红→绿):

  • 新增 5 条锁回归测试在修复前代码上 3 failed(并发阻塞 / 有界重规划 / 超时参数;均 1.5s 内快速失败)→ 修复后 5 passed。
  • OPENVIKING_CONFIG_FILE=/tmp/ovtest/ov.conf timeout 300 uv run --no-sync pytest tests/session/train/test_train_framework.py -q --override-ini addopts="":修复后 39 passed(基线 14 failed / 20 passed;14 条为 main 上既有的 DummyVikingFS 缺 _async_agfs 红,被本修复转绿)。
  • OPENVIKING_CONFIG_FILE=/tmp/ovtest/ov.conf timeout 300 uv run --no-sync pytest tests/session/train/ -q --override-ini addopts="":103 passed / 12 failed;12 条已用 git stash 在干净 HEAD 复跑证实为同集既有环境性失败(2 real-LLM e2e + 10 tau2),与本修复无关。
  • 受害方(并发提交方)回归:uv run --no-sync pytest tests/session/memory/test_streaming_memory_updater.py tests/session/train/test_train_components.py -q:72 passed / 0 failed。
  • uv run --no-sync ruff check openviking/session/train/engine.py openviking/session/train/domain.py tests/session/train/test_fakes.py tests/session/train/test_train_framework.py:All checks passed。
  • 基线具名筛选命令 OPENVIKING_CONFIG_FILE=/tmp/ovtest/ov.conf timeout 300 uv run --no-sync pytest tests/session/train/test_train_framework.py -q -k 'plan or lock' --override-ini addopts="":修复前 0 selected / 34 deselected(既有测试名无匹配,如实记录);修复后同命令 5 passed。

Checklist

  • My code follows the project's coding style
  • I have performed a self-review of my code
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • Any dependent changes have been merged and published

Screenshots (if applicable)

Additional Notes

  • 补丁由 AI 辅助准备并经人工评审(Human Involvement 如上勾选)。
  • 本地环境 ~/.gitconfig 存在 http.proxy https.proxy 误配,HTTPS git 操作需按次 -c http.proxy= 覆盖,与 upstream 无关。

plan_and_apply held the policy-root tree lock across reload ->
policy_optimizer.plan (remote LLM, minutes-long in production traces)
-> apply, so concurrent session commits of the same root blocked until
the 300s pathlock acquisition timeout and failed terminally (volcengine#6095).

Plan against a lock-free snapshot instead, then take the lock only to
re-validate that snapshot before applying. If another writer committed
meanwhile, re-plan against the fresh snapshot (bounded at 2 replans);
once the bound is exhausted, apply the last plan under the lock with a
conflict marker and let the updater's per-item base-content guard
report stale targets as errors instead of silently overwriting them.
PolicySet.lock now takes a timeout_secs parameter (default 300s) and
its docstring no longer promises a wait-indefinitely None mode that
the implementation never had.

Tests: add LockingAGFS (tree-lock fake with real mutual exclusion,
deadline handling and hold-interval recording) plus regression tests
asserting that (1) a 2s plan no longer blocks a concurrent writer
under a 1s lock budget, (2) snapshot conflicts trigger bounded
re-plans followed by a guarded apply against the freshest snapshot,
(3) the uncontended fast path plans once and releases the lock, and
(4) lock timeouts are passed through and leases released on error.
Giving the DummyVikingFS fake the _async_agfs attribute that lock()
requires also fixes the 14 train framework tests that already failed
on unmodified main.

Patch prepared with AI assistance and human review.
@akushonkamen

Copy link
Copy Markdown
Contributor Author

Thanks for the runs! About the one failing check — API & CLI Integration Tests (ubuntu-24.04) ends with 5 failed, 8 passed, 1 skipped, and all five failures are the same server-side error, e.g.:

PROCESSING_ERROR: Failed to generate embedding: Volcengine embedding failed: Error code: 401 - {'error': {'code': 'AuthenticationError', 'message': 'The API key format is incorrect. ...'}}

So the integration environment's Volcengine embedding credentials are being rejected (401 Unauthorized). This PR doesn't touch the embedding/API surface at all — the diff is limited to openviking/session/train/domain.py, openviking/session/train/engine.py and the train tests. The same workflow fails identically on unrelated PRs: run 38811885113 (fix/6130-orphan-running-task-expiry) shows the same 5 failed, 8 passed, 1 skipped with the same 401 signature, so this looks like a CI-side credential issue rather than anything introduced here.

For what it's worth, the checks that do run on this PR are green (check-deps, plugin-tests, plugin-versions; the others skip by path filter), and the unit tests covering this change pass locally on this branch (b6fb190):

uv run python -m pytest tests/session/train/test_fakes.py tests/session/train/test_train_framework.py
→ 39 passed, 0 failed

Happy to help debug the CI credentials if useful — thank you!

@ZaynJarvis ZaynJarvis added the kernel Core server, runtime, storage, retrieval, SDK, and CLI. label Oct 16, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

kernel Core server, runtime, storage, retrieval, SDK, and CLI.

Projects

Status: Backlog

Development

Successfully merging this pull request may close these issues.

[Bug]: V3 policy planning holds the experience tree lock across long LLM calls, blocking concurrent writes

2 participants