Repository navigation
fix(train): plan policy updates outside the policy-root tree lock - #6147
akushonkamen wants to merge 1 commit into
Conversation
plan_and_apply held the policy-root tree lock across reload -> policy_optimizer.plan (remote LLM, minutes-long in production traces) -> apply, so concurrent session commits of the same root blocked until the 300s pathlock acquisition timeout and failed terminally (volcengine#6095). Plan against a lock-free snapshot instead, then take the lock only to re-validate that snapshot before applying. If another writer committed meanwhile, re-plan against the fresh snapshot (bounded at 2 replans); once the bound is exhausted, apply the last plan under the lock with a conflict marker and let the updater's per-item base-content guard report stale targets as errors instead of silently overwriting them. PolicySet.lock now takes a timeout_secs parameter (default 300s) and its docstring no longer promises a wait-indefinitely None mode that the implementation never had. Tests: add LockingAGFS (tree-lock fake with real mutual exclusion, deadline handling and hold-interval recording) plus regression tests asserting that (1) a 2s plan no longer blocks a concurrent writer under a 1s lock budget, (2) snapshot conflicts trigger bounded re-plans followed by a guarded apply against the freshest snapshot, (3) the uncontended fast path plans once and releases the lock, and (4) lock timeouts are passed through and leases released on error. Giving the DummyVikingFS fake the _async_agfs attribute that lock() requires also fixes the 14 train framework tests that already failed on unmodified main. Patch prepared with AI assistance and human review.
|
Thanks for the runs! About the one failing check — So the integration environment's Volcengine embedding credentials are being rejected (401 Unauthorized). This PR doesn't touch the embedding/API surface at all — the diff is limited to For what it's worth, the checks that do run on this PR are green ( Happy to help debug the CI credentials if useful — thank you! |
Description
本 PR 修复训练引擎在策略根目录整树锁内执行远程 LLM 规划导致的并发阻塞(#6095)。
根因:
openviking/session/train/engine.py的plan_and_apply在policy_set.lock()整根树锁内依次执行 reload →policy_optimizer.plan(远程 LLM 调用,生产 trace 中为分钟级)→ apply。并发 session-commit 在openviking/session/train/domain.py的pathlock_acquire_tree(timeout_secs=300.0)(原先硬编码)上等待,等满 300s 后终态失败。(实测基于 HEAD=6006fc34:锁调用在domain.py:84,docstring 中 "timeout=Nonemeans wait indefinitely" 的承诺在:72,实现从未支持 None;工单引用:100为行漂移。)修法:把规划移到锁外——先无锁 reload 出快照并基于快照规划;拿锁后仅做 reload 校验快照仍然一致才 apply;不一致则基于新快照有界重规划(
_PLAN_APPLY_MAX_REPLANS=2);重规划预算耗尽后,带optimistic_apply_after_conflict标记对最新快照守卫应用,由 updater 既有 per-itembefore_content基线守卫把过期目标转为记录错误而非静默覆盖。同时PolicySet.lock增加timeout_secs参数(默认 300s 不变),并修正 docstring 与实现不符的 "None=无限等待" 承诺。Human Involvement
Related Issue
Fixes #6095
Type of Change
Changes Made
engine.py:plan_and_apply改为"锁外快照规划 + 锁内校验后短临界区应用";快照冲突时基于新快照有界重规划(上限_PLAN_APPLY_MAX_REPLANS=2);预算耗尽后带optimistic_apply_after_conflict标记守卫应用,过期目标由既有 per-item 基线守卫报错而非静默覆盖。domain.py:PolicySet.lock新增timeout_secs关键字参数(默认300.0,行为不变),锁调用透传该参数;docstring 删除实现从未支持的 "timeout=None无限等待" 承诺,改为说明临界区应保持短小、远程 LLM 规划须在锁外执行。tests/session/train/test_fakes.py:新增LockingAGFS(具备真实互斥、deadline 处理与持锁区间记录的树锁 fake);DummyVikingFS补上lock()所需的_async_agfs属性。tests/session/train/test_train_framework.py:新增 5 条锁回归测试——(1) 2s 规划不再在 1s 锁预算内阻塞并发写者;(2) 快照冲突触发有界重规划并对最新快照守卫应用;(3) 无竞争快路径仅规划一次并释放锁;(4) 锁超时参数透传、出错时释放租约。Testing
本地验证(红→绿):
OPENVIKING_CONFIG_FILE=/tmp/ovtest/ov.conf timeout 300 uv run --no-sync pytest tests/session/train/test_train_framework.py -q --override-ini addopts="":修复后 39 passed(基线 14 failed / 20 passed;14 条为 main 上既有的DummyVikingFS缺_async_agfs红,被本修复转绿)。OPENVIKING_CONFIG_FILE=/tmp/ovtest/ov.conf timeout 300 uv run --no-sync pytest tests/session/train/ -q --override-ini addopts="":103 passed / 12 failed;12 条已用git stash在干净 HEAD 复跑证实为同集既有环境性失败(2 real-LLM e2e + 10 tau2),与本修复无关。uv run --no-sync pytest tests/session/memory/test_streaming_memory_updater.py tests/session/train/test_train_components.py -q:72 passed / 0 failed。uv run --no-sync ruff check openviking/session/train/engine.py openviking/session/train/domain.py tests/session/train/test_fakes.py tests/session/train/test_train_framework.py:All checks passed。OPENVIKING_CONFIG_FILE=/tmp/ovtest/ov.conf timeout 300 uv run --no-sync pytest tests/session/train/test_train_framework.py -q -k 'plan or lock' --override-ini addopts="":修复前 0 selected / 34 deselected(既有测试名无匹配,如实记录);修复后同命令 5 passed。Checklist
Screenshots (if applicable)
Additional Notes
~/.gitconfig存在http.proxy https.proxy误配,HTTPS git 操作需按次-c http.proxy=覆盖,与 upstream 无关。