OrchOpt: Validation-Gated Architecture Search for Self-Organizing Multi-Agent Orchestration

Abstract

Multi-agent architectures are typically hand-designed, generated in one shot, or evolved through unconstrained code search. None of these treats the number of agents as a searchable parameter, and none can discover that a single agent is optimal. We argue that agent count and coordination topology deserve the same optimization discipline used for neural architectures: bounded edits, held-out validation gating, and bidirectional search that adds and removes agents without information loss. OrchOpt is a validation-gated architecture search over a unified design space in which single-agent and multi-agent configurations compete directly, with bidirectional traversal enabled by knowledge-preserving merge. A proposer model turns failure analysis into bounded merge, split, and skill edits on the current architecture, and an edit is accepted only when it strictly improves validation. Because an absorbed agent’s expertise becomes an activatable skill, merging loses no information and the search stays fully reversible at no additional inference cost. Across nine benchmarks spanning math, knowledge QA, code generation, reading comprehension, multi-hop reasoning, negotiation, and research, the optimizer simplifies to 1 agent on 4 domains and elaborates to 2–3 agents on 5 domains. It finds architectures scoring +20pp over ADAS on MGSM, +16.7pp over AFlow on HotpotQA, and +27.5pp over MARBLE’s published best topology on Bargaining, all within 7–42 candidate evaluations.