MixUA Scopes Agent Memory Across Users, Agents, and Privacy Boundaries

Summary:MixUA introduces a hierarchical memory architecture for multi-user, multi-agent systems. It distills reusable lessons from independent interaction trajectories, then assigns each lesson to one of four visibility scopes: a specific user-agent pair, a user, an agent, or the global environment. This design separates useful cross-agent experience from private user facts instead of writing every insight into one shared memory store. The accompanying CrossLearnBench combines established agent tasks,

---

Hierarchical Scopes Instead of One Shared Store

MixUA addresses a problem often overlooked in multi-user, multi-agent systems: experience is not automatically safe or useful to share globally. A lesson may apply only to one user-agent combination, or it may benefit one specific agent without exposing any personal information. Writing all such insights into a single global store improves reuse at the cost of relevance and privacy.

MixUA first distills reusable lessons from independent user-agent trajectories. It then assigns each lesson to one of four scopes:

1. User-agent pair: visible only to that user interacting with that agent;
2. User: reusable across agents but restricted to one user;
3. Agent: reusable across users but restricted to one agent;
4. Global: available to all participants, intended for non-identifying task knowledge.

The important architectural choice is that sharing decisions are made during memory creation rather than postponed to keyword filtering during retrieval. A production implementation would attach ownership, scope, and disclosure metadata to every memory and enforce authorization before retrieval. Filtering sensitive text only after generation is weaker because restricted information may already have entered the model's context or application logs.

How Multi-Granularity Extraction Works

From an engineering perspective, the pipeline has four layers:

• Trajectory collection: captures user requests, agent actions, tool results, and task feedback;
• Experience distillation: compresses trajectories into reusable lessons while retaining triggers, context, and outcomes;
• Scope assignment: estimates who contributed the lesson, who can benefit from it, and what may be disclosed;
• Controlled retrieval: filters visible memories using the current user, agent, and task before injecting them into planning.

The limiting factor is likely to be the quality of distillation and scope classification, rather than the number of stored memories. Promoting a private preference to global scope can create a direct privacy violation; over-localizing a generic skill weakens cross-user reuse. Production systems therefore need layered controls: policy-level authorization for every retrieval path and independent privacy probes for continuous evaluation of classification errors.

Benchmark Design and Results

The accompanying CrossLearnBench tests three capabilities together: completion of established agent tasks, retention of user preferences, and prevention of private facts from entering readable scopes. The experiment covers five users and seven agents, including tasks previously attempted only by other users on other agents.

The paper reports that MixUA:

• improves task pass rates by as much as 16 percentage points over no memory;
• improves them by as much as five percentage points over a single global memory;
• retains up to 90% of previously stated preferences;
• exposes planted private facts in forbidden scopes at a rate of no more than 0.02%.

These findings suggest that controlled sharing can outperform unrestricted global memory on both utility and privacy grounds. A global store may reduce retrieval precision while increasing the blast radius of an access-control error. The five-point gain over global memory indicates that scoping can improve task relevance, not merely reduce disclosure.

The abstract is insufficient for strong statistical claims. It does not provide sample size, confidence intervals, significance tests, or per-task variance. Therefore, the maximum improvement may not represent the average effect. Likewise, a 0.02% leakage rate is low but nonzero; the denominator and probe design matter, particularly for indirect leakage through paraphrases, prompt injection, or compositional inference.

Engineering Trade-offs

MixUA avoids retraining the foundation model, but shifts costs into memory infrastructure:

• Storage and retrieval: lessons may be duplicated across scopes, while ownership and permission metadata complicate indexing. Filtering typically requires metadata joins, policy checks, or partitioned stores rather than vector similarity alone.
• Distillation overhead: summarizing every interaction adds model calls, tail latency, and opportunities for incorrect summaries to contaminate long-term memory.
• Isolation risk: scope enforcement must cover caches, logs, vector databases, backups, and developer tools. A missing check at any layer can bypass access control.
• Cold starts and sparsity: dedicated user-agent scopes can accumulate too little evidence to be useful, requiring conservative promotion and fallback rules.
• Deletion and revocation: removing a user preference must also address derived summaries, embeddings, caches, and logs. Deleting a database row alone is insufficient.

MixUA is therefore better viewed as a memory-governance architecture than a drop-in replacement for RAG or an experience store. It usefully integrates cross-user reuse with privacy boundaries, but production deployment still requires formal access control, audit logs, and verifiable deletion.

Deployment Guidance

Developers integrating a MixUA-style system should:

1. Define user, agent, user-agent pair, and global scopes, and store source, beneficiary, sensitivity, and retention metadata with every memory;
2. Enforce permission filtering at retrieval time, never relying on the generation model to self-censor in its prompt;
3. Evaluate task success, preference retention, unauthorized recall, and forbidden-scope leakage in one benchmark;
4. Assign low-confidence distilled lessons only to private scopes;
5. Run deletion drills across vectors, caches, logs, and derived experiences;
6. Use an independent audit model plus human review to evaluate scope assignment, rather than letting the same model write and validate every policy.

For enterprise agent platforms, the priority is not merely how much global experience improves task scores. The system must make authorization failures controllable, leakage auditable, and user deletion enforceable before cross-user memory is used in long-running production workloads.

---

Source: Hugging Face Daily Papers (2026-11-07)

— END —

Tags: #Multi-Agent Memory Follow #User Isolation Follow #Privacy Boundaries Follow #Experience Reuse Follow #CrossLearnBench Follow