Skip to content

feat(cs): add async multi-turn agentic rollout - #361

Draft
Yunnglin wants to merge 17 commits into
dev-swift-v5from
cs-async-rl
Draft

Yunnglin wants to merge 17 commits into
dev-swift-v5from
cs-async-rl

Conversation

@Yunnglin

@Yunnglin Yunnglin commented Oct 31, 2026 •

Copy link
Copy Markdown
Collaborator

Features and Fixes

  • Shared core HTTP multi-turn rollout, asynchronous entry points, and resource borrowing isolation.
  • AgentENV GRPO cookbook, independent task scoring, and dedicated acceptance testing entry points.
  • Fixed W1–W7: synchronous cancellation aggregation, initialization finalization, template paths, independent evaluation/seed, tool cancellation phase, wait-for-exit handling, and evidence determination.
  • Preserve absent multimodal fields in text-only trajectories.
  • Synchronize gradients on completed accumulation steps, fixing GA=2 updates that previously returned without changing weights.

Validation update — 9d918c0e

  • Current-HEAD Transformers/Megatron × Twinkle/Tinker × SFT/DPO/GRPO: 12/12 passed, 0 skipped.
  • SFT verifies LoRA + optimizer save/resume and continued training for both clients on both backends.
  • Both backends confirm training instances STOPPED and training_resources_released=true; no validation processes remain.
  • The GA=2 fix is also covered by a minimal LoRA update probe and 110 related regression tests.

Evidence:

  • .qoder/evidence/d13-aeb73daf/matrix-9d918c0e-transformers/run.json
  • .qoder/evidence/d13-aeb73daf/matrix-9d918c0e-megatron/run.json
  • .qoder/evidence/d13-aeb73daf/scheduler-matrix-9d918c0e/scheduler.json

Pending

  • Post-training sync/async parity investigation and the complete OpenEnv pipeline/learning/six-stage natural-cancellation suite are pending by user request.
  • CI/lint cleanup is outside this round's scope by user request.
  • OpenEnv functional coverage does not replace the AgentENV isolation/security gate.
  • The 12/12 training matrix must not be interpreted as full D13 completion or completion of the pending items.

@Yunnglin
Yunnglin changed the base branch from main to dev-swift-v5 November 2, 2026 04:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant