arXiv Releases SWE-Duel: A Self-Scaling Arena for Coding Agents

Summary:arXiv introduces SWE-Duel, a self-scaling benchmark arena for coding agents. Unlike traditional static benchmarks, SWE-Duel dynamically generates tasks and evaluation criteria to prevent content leakage and saturation over time. The arena employs a mirrored Red-Blue turn system where agents alternately program and review code, simulating real-world collaborative and competitive development environments. Experiments demonstrate that SWE-Duel effectively distinguishes agent capabilities and provid

---

Core Mechanisms and Technical Innovations

SWE-Duel achieves the following through its design:

1. Dynamic Task Generation: Tasks and evaluation criteria are generated at evaluation time, preventing the leakage and saturation issues inherent in traditional static benchmarks.
2. Mirrored Red-Blue Turn System: The arena employs a mirrored Red-Blue turn system where agents alternate between programming and reviewing, simulating real-world collaborative and competitive development environments.
3. Multi-Stage Evaluation: Each turn includes feature implementation, bug injection, code review, and bug repair stages, ensuring a comprehensive and accurate evaluation.
4. Self-Repair Mechanism: The ability of agents to repair bugs verifies the feasibility of tasks and the robustness of agents.

Engineering Trade-offs and Performance

• Advantages:

  • Prevents task leakage and saturation, providing a more sustainable evaluation standard.
  • Simulates the complexity of real-world development environments, enhancing evaluation authenticity.
  • Comprehensive evaluation through multi-stage assessment, examining both programming and reviewing capabilities of agents.

• Challenges:

  • Dynamic task generation increases system complexity, requiring efficient algorithms and resource allocation.
  • The Red-Blue turn system may extend evaluation time, necessitating process optimization for efficiency.

Experimental results demonstrate that SWE-Duel effectively distinguishes agent capabilities. For instance, GPT-5.5 excels in defect repair, fixing 98.3% of the defects it reviews, while Claude Opus 4.8 is the strongest task author with a 27.3% Red win rate. Additionally, within the same model family, tasks authored by the newest version defeat a fixed-reference solver far more often than those of older versions (42.9% vs. 4.0%), providing preliminary evidence that task difficulty scales with authoring capability.

Developer Deployment and Implementation Recommendations

1. Model Training and Evaluation: Developers are encouraged to use SWE-Duel as a benchmark platform for training and evaluating coding agents to obtain performance metrics that are more aligned with real-world application scenarios.
2. Task Design: Design diverse task types based on specific application scenarios to ensure the adaptability and robustness of agents in different contexts.
3. Resource Optimization: Optimize system resource allocation for dynamic task generation and the Red-Blue turn system to improve evaluation efficiency.

Conclusion

SWE-Duel offers an innovative solution for the training and evaluation of coding agents, showcasing the potential of self-scaling benchmark platforms. Its dynamic task generation and Red-Blue turn system provide new perspectives and methodologies for the AI research community.

---

Source: ArXiv NLP/LLM (cs.CL) (2026-10-12)

— END —

Tags: #SWE-Duel Follow #Coding Agents Follow #Self-Scaling Benchmark Follow #AI Arena Follow