Agent-R² Trains a 4B Office Agent from Real-World Work Data, Outperforming a 235B Model
Summary:Agent-R² introduces an end-to-end synthesis framework for realistic productivity tasks. Its pipeline starts from occupational seeds, workflow evidence extracted from corpora, and authentic web-sourced attachments, producing 4,000 multi-format task instances from 10,000 web-corpus candidates. The resulting Agent-R²-4B model is trained with supervised fine-tuning and rubric-based reinforcement learning. Despite using only 4,000 instances, the 4B model reaches 51.0% on GDPval and 14.8% on OfficeQA
A Real-File, Deliverable-Level Post-Training Loop
Agent-R² departs from the common pipeline in which an LLM specifies workflows and another LLM fabricates the associated files. Instead, the framework anchors synthesis in occupational seeds, workflow evidence extracted from corpora, and authentic attachments sourced from the web. The team filters roughly 10,000 occupational candidates from a web corpus and distills them into 4,000 task instances, each combining attachments in multiple formats with fine-grained, deliverable-level rubrics.
This addresses a central weakness in office-agent evaluation. A correct answer is not necessarily a completed task. An agent may need to inspect PDFs, spreadsheets, and documents, preserve constraints over a long workflow, and produce a file that another person can use directly. Agent-R² therefore moves the target from answer matching to artifact acceptance, teaching the model when to read evidence, how to construct outputs, and when to verify them.
A 4B Model Surpasses a 235B Baseline
The researchers train Agent-R²-4B on the 4,000 instances using supervised fine-tuning and rubric-based reinforcement learning. SFT establishes the initial task trajectory and output behavior, while the rubric reward optimizes the quality of the final artifact.
On GDPval, Agent-R²-4B scores 51.0%, compared with 48.0% for Qwen3-235B-A22B-Thinking-2507. On OfficeQA, the respective scores are 14.8% and 13.8%. The smaller model wins on both benchmarks, suggesting that parameter count is not the only determinant of complex office work and that alignment between training data and the actual file environment may matter more.
This should not be interpreted as a general capability advantage of 4B over 235B. The occupational coverage, rubric design, task mix, and inference configuration can materially affect the comparison. The absolute scores also remain modest, so beating a large baseline does not mean that reliable, unattended office automation has been achieved.
Rubric Rewards Target the Final Artifact
Office agents generate spreadsheets, documents, reports, and other artifacts rather than plain text. If rewards depend only on string matching or an unconstrained LLM judge, an agent may produce fluent but incorrect outputs with missing constraints, unsupported claims, or invalid formatting. Agent-R² uses fine-grained rubrics to decompose deliverable quality into verifiable dimensions, allowing reinforcement learning to distinguish completed steps from genuinely usable work products.
The benefit is a tighter alignment between optimization and task completion. The cost is that rubric construction and evaluation become part of the training pipeline. An imperfect evaluator can misjudge formatting, overlook subtle constraints, or favor particular document styles. If the agent learns to infer and exploit rubric expectations, it may optimize for the scorer rather than for actual productivity.
Authentic Attachments Improve Grounding but Add Noise and Risk
The analysis also reports that Agent-R² mitigates shortcut behavior in which an agent submits a superficially plausible deliverable before sufficiently grounding, constructing, or verifying it. The trained model shows more sustained exploration and verification. Authentic multi-format files supply concrete training signals for this behavior because useful output depends on reading those files rather than inventing their contents.
Real files are not automatically clean training data. Web attachments may contain corrupted formats, duplicated records, embedded prompt injection, private information, or copyright-restricted material. A secure pipeline must treat document contents as untrusted data, strip or quarantine active instructions, preserve provenance and licensing records, and separate evaluation tasks by occupation and file type.
Engineering Trade-Offs: Dense Data Is Not Free Data
The dataset contains only 4,000 instances, far fewer than a conventional instruction-tuning corpus, but every instance requires real attachments, task construction, and rubric validation. Distillation also involves candidate filtering, occupational mapping, workflow extraction, and artifact packaging, so this is not equivalent to generating a large batch of question-answer pairs.
A 4B model lowers training and private-deployment requirements, yet a complete office-agent system still pays substantial inference costs for long contexts, multi-format parsing, tool calls, and artifact verification. Winning a benchmark does not establish lower total token consumption, latency, or retry count in production. Real deployments also need parallel file parsing, tool sandboxes, resumable execution, versioned outputs, and human approval gates.
Practical Recommendations for Developers
Teams reproducing this approach should structure each task around five layers: input files, constraints, tool trajectory, final artifact, and acceptance rubric. They should begin with domains whose outputs can be checked automatically, such as spreadsheet aggregation, document extraction, compliance checks, and multi-file comparison. For open-ended deliverables, combine deterministic validation, executable checks, and sampled human review.
Evaluation should separately track final-file correctness, tool-call cost, end-to-end latency, retry count, and human rework time. Without those measurements, it is difficult to determine whether the gains from real-source data survive the longer execution chains and more expensive file processing required in production.
Agent-R²’s main contribution is not a new foundation-model architecture. It shows that occupational data, authentic files, and deliverable-level rewards can form a low-sample, high-density post-training path for office agents. The 4B model's victory over a 235B baseline is a meaningful signal, but the stronger conclusion still requires independent validation across occupations, file formats, and agent execution frameworks.
— END —Source: Hugging Face Daily Papers (2026-11-05)
Tags: #Agent-R² #Office Agents #Real-Source Data Synthesis #Rubric-Based Reinforcement Learning #Multi-Format Files
Editorial & Fact-Checking Note: This article is compiled from primary research, official release documentation, and source papers by the ZICQ Newsroom pipeline with automated entity verification and human editorial review. If you notice any technical inaccuracy, please submit a correction via our corrections policy or email our editorial desk directly.
Community Comments