arXiv Releases Constrained Agentic RL Model for Web Agents, Boosting Task Efficiency with Multi-Cost and Risk Control

Summary:arXiv introduces a novel constrained agentic reinforcement learning model for web agents, unifying page access, search requests, and external API calls into a unified decision-making framework with associated costs. By incorporating cost budget constraints and tail risk control, the model addresses the shortcomings of traditional methods, such as 'high success but high cost' or 'low risk but conservative failure.' Experiments demonstrate that the method reduces cost-per-success (C-PS) by 22%-31%

---

Technical Mechanism Analysis

The research introduces a constrained agentic reinforcement learning model designed to address the shortcomings of traditional methods in web agent interactions. The core mechanisms include:

1. Unified Decision Framework: Integrating page access, search requests, and external API calls into a unified long-term decision-making framework with associated costs.
2. Multidimensional Cost Vector: Constructing a cost vector comprising cumulative request count, total latency, and failure penalties, achieving budget compliance through Lagrange dual updates.
3. Tail Risk Control: Employing a Conditional Value at Risk (CVaR) risk term to suppress excessive exploration of high-failure-probability paths, thereby achieving an adaptive balance among 'completion rate, cost, and risk.'

Engineering Trade-offs and Experimental Performance

Experiments were conducted across 30-70 site/page templates and 800-1,500 end-to-end web tasks, including information extraction, price comparison, form submission, and cross-page navigation. The results demonstrate:

• A reduction in cost-per-success (C-PS) by 22%-31% and failure rates by 18%-26% at equivalent completion rates.
• An increase in task completion rates by 10%-16% under fixed budgets.

These findings highlight the necessity and effectiveness of constraint modeling for practical web agent deployment.

Developer Deployment and Implementation Recommendations

1. Multimodal Task Support: The model is suitable for a variety of web tasks, including information extraction, navigation, and API calls. Developers can customize it based on specific requirements.
2. Cost and Risk Control: It is recommended to adjust model parameters according to actual budget and risk tolerance to achieve optimal performance.
3. Performance Optimization: Given the involvement of multidimensional cost vectors and tail risk control, it is advisable to run the model in an environment with sufficient hardware resources to ensure the best inference efficiency.

Conclusion

The study demonstrates the potential of constraint modeling in agentic reinforcement learning, particularly in the context of practical web tasks. By effectively balancing completion rate, cost, and risk, the model provides a new solution for efficient execution of intelligent agents in complex web environments.

---

Source: ArXiv Machine Learning (cs.LG) (2026-10-21)

— END —

Tags: #Intelligent Agent Follow #Reinforcement Learning Follow #Web Agent Follow #Multi-Cost Control Follow #Tail Risk Follow