MAS-Orchestra: Understanding and Improving Multi-Agent Reasoning Through Holistic Orchestration and Controlled Benchmarks
MAS-Orchestra is an RL training framework with a function-calling abstraction for holistic multi-agent orchestration — composing an entire MAS at each step rather than building it incrementally. We also build MASBench, a controlled benchmark for studying when multi-agent systems outperform single-agent systems. MAS-Orchestra outperforms GPT-5 and Claude-Sonnet-4.5 by up to 23% across 5 benchmarks with a 10× efficiency gain.
- [05/2026] MAS-Orchestra is accepted to ICML 2026!
- [04/2026] We present the Live Demo! Start vibe-coding your MAS today! [Live Demo]
- [01/2026] We present the MAS-Orchestra [Project Page | Paper | Code]
Accuracy vs. cost Pareto front. MAS-Orchestra achieves Pareto-optimal performance with the highest accuracy at low cost.
MAS-Orchestra achieves state-of-the-art performance across both IID and OOD benchmarks while maintaining Pareto-optimal cost efficiency.
| Method | IID Tasks | OOD Task GPQA |
|||
|---|---|---|---|---|---|
| AIME24 | AIME25 | HotpotQA | BrowseComp+ | ||
| Standalone Agents | |||||
| CoTAgent | 50.00 | 45.00 | 33.56 | 1.12 | 60.54 |
| SCAgent | 57.50 | 51.67 | 35.50 | 0.75 | 62.88 |
| DebateAgent | 62.08 | 57.50 | 36.88 | 0.81 | 64.14 |
| ReflexionAgent | 60.83 | 50.42 | 36.63 | 1.00 | 62.37 |
| DeepResearchAgent | — | — | 46.44 | 8.56 | — |
| SoTA Inference-time Orchestration | |||||
| AFlow | 62.50 | 53.33 | — | — | 65.43 |
| MaAS | 32.50 | 40.83 | — | — | 40.78 |
| MAS-Zero | No valid MAS generated with 7B orchestrator | ||||
| SoTA Public Training-time Orchestration | |||||
| MAS-GPT | 58.75 | 43.33 | — | — | 63.51 |
| ToolOrchestra | 23.33 | 11.25 | 37.44 | 1.38 | 29.80 |
| SoTA LLM as Orchestrator | |||||
| GPT-5 | 55.00 | 47.72 | 25.87 | 0.50 | 59.01 |
| Claude-Sonnet-4.5 | 45.56 | 35.00 | 38.00 | 0.50 | 21.72 |
| Ours | |||||
| MAS-Orchestra | 66.25 | 61.25 | 49.00 | 11.00 | 65.21 |
Performance comparison across IID and OOD benchmarks. MAS-Orchestra achieves the best results on all tasks.
conda create -n mas-orchestra python=3.10
conda activate mas-orchestra
apt update && apt install -y wget curl
cd ./verl
./install.sh
pip install --no-deps -e .
pip install ray==2.49.2 --force-reinstall
pip install protobuf==4.25.8 --force-reinstall
pip install together
pip install math-verify[antlr4_13_2]
pip install antlr4-python3-runtime==4.9.3
pip install langchain-core langchain-together langchain-community duckduckgo-search tavily-python pydantic ddgs langchain_brightdata bs4
pip install pyserini faiss-gpu
pip install git+https://github.com/texttron/tevatron.git
| Task | Model |
|---|---|
| Math (AIME) | harmony-grpo-7b-global-step-180 |
| HotpotQA | harmony-medium-grpo-7b-hotpot-global-step-250 |
| BrowseComp+ | harmony-medium-grpo-7b-browse-comp-plus-global-step-140 |
The trained orchestrators can also be called via API without downloading. Check out the notebook below for usage:
&color=blue&style=for-the-badge){kind=icon}
export OPENAI_API_KEY={YourKey}
export TOGETHER_API_KEY={YourKey}
export WANDB_API_KEY={YourKey}
LOG_FILE={YourLogFile}
python -u -m mas_r1_reasoner.main_mas_r1 \
--config-path=configs \
--config-name=grpo_trainer \
data.max_prompt_length=15000 \
data.max_validation_prompt_length=15000 \
data.val_files=data/browse_comp/test_subset_200.parquet \
data.train_files=data/browse_comp/train_subset_1066.parquet \
azr.mas_r1.use_llm_judge=True \
data.raw_data=True \
data.train_batch_size=64 \
actor_rollout_ref.rollout.n=32 \
azr.mas_r1.execution_success_weight=0.0 \
azr.mas_r1.final_answer_weight=1.0 \
azr.mas_r1.agent.model_name=gpt-oss-120b\
azr.mas_r1.multiply_processes=0 \
azr.mas_r1.max_ray_workers=1 \
azr.problem_type=harmony_medium \
azr.mas_r1.agent.init_archive=['COT','COT_SC','Reflexion','LLM_debate','WebSearch'] \
trainer.val_before_train=True \
trainer.test_freq=5 \
trainer.save_freq=10 \
actor_rollout_ref.model.path=Qwen/Qwen2.5-7B-Instruct \
trainer.experiment_name=harmony_medium_grpo_7b_gpt_oss_120b_browse_comp_plus \
$@ 2>&1 | tee -a "$LOG_FILE"MASBench is a controlled benchmark that characterizes tasks along five structural axes to rigorously study when and why multi-agent systems outperform single-agent systems.
| Axis | Definition |
|---|---|
| Depth | Length of the longest dependency chain |
| Horizon | Number of intermediate sub-tasks whose answers are needed |
| Breadth | Maximum in-degree, i.e., maximum dependencies of a sub-task |
| Parallel | Number of independent sub-task components in the task |
| Robustness | Number of sub-tasks with adversarial attacks |
The benchmark covers all five axes with axis values ranging from 2 to 12, and provides axis-specific training and test splits. The dataset is available on Hugging Face.
Browse real, generated multi-agent designs produced by MAS-Orchestra. Each example shows the full orchestration trace — how the orchestrator decomposes a task, selects sub-agents, and aggregates their outputs.
Highlights from the case studies:
- AIME24 (Low DoM): MAS-Orchestra learns to delegate entirely to a single strong sub-agent (100% delegation after 20 training steps), dynamically selecting ReflexionAgent or DebateAgent — the best-performing standalone baselines.
- BrowseComp+ (High DoM): MAS-Orchestra generates substantially more sub-agents, invoking SearchAgent with 3–4 parallel search processes per question.
- General Pattern: MAS-Orchestra dynamically adapts to each task by proposing MAS designs that align with the underlying sub-task structure and delegating execution to the most effective agent configurations.
🗺️ Over the past year, we've built a connected body of work on multi-agent systems — spanning the framework, inference-time and training-time methods, verification, analysis, and a live demo. Explore how it all fits together in our MAS framework and roadmap.
- MAS-Zero: Designing Multi-Agent Systems with Zero Supervision — an inference-time self-refinement framework for automatic MAS design.
- MAS-ProVe: Understanding the Process Verification of Multi-Agent Systems — analysis of process verification for multi-agent systems.
- SkillOrchestra: Learning to Route Agents via Skill Transfer — skill-based agent routing.
- OrchRM: Reward Modeling for Multi-Agent Orchestration — a Bradley-Terry orchestration-level reward model to power best-of-N test-time scaling and reward-guided orchestrator training.
- IlluMAS: The Illusion of Multi-Agent Advantage — a cost-controlled study showing automatic MAS often fail to beat SAS, introducing the SMFR diagnostic benchmark with a strong expert-designed MAS baseline.
- LLM Reasoning Survey: A Survey of Frontiers in LLM Reasoning: Inference Scaling, Learning to Reason, and Agentic Systems.
If you find MAS-Orchestra helpful, please consider starring this repo and citing our work. We would be very grateful!
@inproceedings{Ke2026MASOrchestra,
title = {MAS-Orchestra: Understanding and Improving Multi-Agent Reasoning Through Holistic Orchestration and Controlled Benchmarks},
author = {Zixuan Ke and Yifei Ming and Austin Xu and Ryan Chin and Xuan-Phi Nguyen and Prathyusha Jwalapuram and Semih Yavuz and Caiming Xiong and Shafiq Joty},
booktitle = {Proceedings of the International Conference on Machine Learning (ICML)},
year = {2026},
}This project received help from many researchers at Salesforce AI Research. We also thank the authors of verl for their excellent contributions to the community!
Feel free to contact Zixuan Ke via email: zixuan.ke@salesforce.com