arXiv Introduces SafeWebWorld: Benchmark for Evaluating Safety Sufficiency in Web World Models

Summary:arXiv has introduced SafeWebWorld, a benchmark designed to evaluate the safety sufficiency of AI agents in web safety domains. The benchmark assesses agents' abilities to predict future actions, preserve critical evidence, and make safe decisions across five web safety domains. The study reveals that while current models demonstrate high semantic plausibility in predictions, they struggle with complex safety decisions, showing significant deficiencies in authorization sensitivity and the ability

---

Background and Motivation

In recent years, the application of AI agents in web environments has become increasingly prevalent, making the assurance of their safety decision-making capabilities in complex scenarios a critical research focus. While existing web world models have made progress in predicting the consequences of actions, the semantic plausibility of these predictions does not always translate into correct safety decisions. For instance, in a human evaluation of 3,000 predictions, 86.1% were deemed semantically plausible, yet the models exhibited significant asymmetry in handling authorization behaviors, combining low false-safe execution rates with poor recall of clarifiable ASK cases and a strong preference for BLOCK actions.

SafeWebWorld Benchmark

SafeWebWorld aims to address these issues by evaluating AI agents' safety decision-making capabilities through the following three aspects:

1. Action-conditioned future prediction: Assessing the agent's ability to predict the consequences of future actions.
2. Policy-decisive evidence preservation: Evaluating the agent's capability to preserve critical evidence during prediction.
3. Policy-conditioned decision-making: Assessing the agent's ability to make safe decisions under different policy conditions.

The study conducted tests across five web safety domains (authentication, privacy protection, data integrity, access control, and malware protection) and found that while evidence preservation improved substantially under a controlled Security-Aligned Supervised Fine-Tuning protocol, authorization-sensitive decisions deteriorated.

Key Findings and Challenges

1. Insufficient handling of authorization behaviors: The models showed significant asymmetry in handling authorization behaviors, with a strong preference for BLOCK actions and poor recall of clarifiable ASK cases.
2. Challenges in decision robustness: Although frozen-score recalibration can partially recover authorization-sensitive scores, this recovery is backbone-dependent and does not translate into robust fine-grained authorization behavior.
3. Need for comprehensive evaluation: Plausible prediction, evidence preservation, and aggregate recovery are each insufficient on their own to establish robust authorization.

Technical Mechanisms and Trade-offs

SafeWebWorld employs an evaluation framework based on action-conditioned prediction and evidence preservation, and uses Latent Authorisation Recovery (LAR) to assess the model's recovery capabilities. LAR, through frozen-score recalibration and paired recovery, demonstrates the potential for recovering authorization-sensitive score structures. However, this recovery does not always translate into robust fine-grained authorization behavior, indicating that there is still significant room for improvement in the safety decision-making capabilities of AI agents.

Developer Recommendations

1. Multi-dimensional evaluation: When evaluating the safety decision-making capabilities of AI agents, consider multiple dimensions such as semantic plausibility, evidence preservation, and authorization-sensitive decisions.
2. Model architecture optimization: To address the insufficiency in handling authorization behaviors, consider optimizing the model architecture, such as introducing more complex attention mechanisms or reinforcement learning strategies.
3. Continuous monitoring and updates: The safety decision-making capabilities of AI agents need to be continuously monitored and updated to address evolving web security threats.

---

Source: ArXiv AI (cs.AI) (2026-11-02)

— END —

Tags: #arXiv Follow #AI Safety Follow #Agentic Follow #Web Security Follow #Benchmark Follow