ArXiv Releases Daydreaming: Stealing Hidden Agent Skills via Black-Box Task Interaction
Summary:ArXiv introduces Daydreaming, a novel attack method that steals multi-file skills through black-box task interactions without directly requesting the skill or reconstructing its text. By adaptively crafting tasks to distinguish hidden behaviors, Daydreaming recovers 86.8% of the original skill's capability across 7 skills and 4 victim models, outperforming SigLeak by nearly 4x. This research highlights the limitations of current disclosure defenses and underscores the risks of functional reconst
Background and Motivation
Agent skills bundle instructions, reference data, and executable helpers, enabling general agents to perform specialized tasks. Hosted providers can keep these files secret while selling access to task results, making the skill itself a valuable target. Existing disclosure defenses can block requests asking for the skill or reproducing its text, but they cannot prevent customers from submitting the tasks the service is built to complete.
Daydreaming Attack Method
Daydreaming is an execution-only attack that steals multi-file skills through black-box task interactions. It does not ask the victim to reveal the skill or grade a reconstruction but instead adaptively crafts tasks whose results distinguish possible hidden behaviors. The steps include:
- Behavior Testing: Testing individual behaviors to identify potential hidden patterns.
- Design Selection: Using attacker-controlled shadow agents to choose the best design.
- File Completion: Completing each file using stored victim results and local execution checks.
Experimental Results
The study formalizes three nested threat levels of access as Differential, Trace, and Output, focusing on the Output level, where the attacker sees only the final response and returned files. Across 7 skills and 4 victim models, Daydreaming recovers 86.8% of the original skill's capability at the Output level, outperforming SigLeak by nearly 4x. Even with disclosure defenses enabled, Daydreaming produces installable skills using a median of 32 victim calls per skill.
Conclusion and Implications
The results demonstrate that hiding skill files and filtering direct disclosure do not, by themselves, prevent functional reconstruction through normal use. This poses new challenges for the security and defense mechanisms of agent skills, emphasizing the need for more comprehensive protection strategies.
Recommendations for Developers
- Enhance Defense Mechanisms: Developers should consider more complex defense strategies, such as behavior analysis, anomaly detection, and dynamic access control, when designing agent skills.
- Multi-layered Security Measures: Combine multiple security measures, such as encryption, access control, and audit logs, to prevent skill leakage.
- Continuous Monitoring and Updates: Regularly monitor agent behavior and update defense mechanisms to address new attack methods.
— END —Source: ArXiv cs.AI (2026-08-27)
Tags: #Agent Security #Black-Box Attack #Skill Theft #ArXiv #AI Safety
Editorial & Fact-Checking Note: This article is compiled from primary research, official release documentation, and source papers by the ZICQ Newsroom pipeline with automated entity verification and human editorial review. If you notice any technical inaccuracy, please submit a correction via our corrections policy or email our editorial desk directly.
Community Comments