Auriga Agent Harness Closes the Robot Planning-to-Execution Gap
Summary:Auriga is a task-agnostic agent harness for zero-shot embodied manipulation. Rather than asking a vision-language model to act directly as a robot policy, it maintains an explicit task belief, synthesizes multi-step Python programs, and routes actions through a calibrated control gateway with 3D measurements, dual-arm finger references, trajectory previews, stage tolerances, and grasp-bound checks. The frozen harness reaches a 57.7% success rate on RoboDojo and 73.2% on RoboCasa365. Its key con
Core Idea: Separate Task Decisions from Physical Execution
Auriga does not treat a vision-language model as a direct end-to-end robot policy. Instead, the model acts as a task planner over an explicit task belief and expresses actions through atomic tool calls or dynamically synthesized multi-step Python programs. Auriga translates those intentions into constrained robot actions.
The paper identifies the same bottleneck on RoboDojo. With the reference control interface, GPT-6 Astra achieves a 22.5% success rate. When arm execution is removed and only high-level task decisions are evaluated, success rises to 76.2%. The report therefore attributes most of the failure to the transition from planning to execution rather than to task comprehension alone. Auriga claims to close 68.3% of that planning-execution gap in zero-shot manipulation without target-task trials or resets.
Execution Architecture: A Single Admission Gateway
Auriga places a structured execution layer between the foundation model and the robot:
- An explicit task belief tracks goals, stages, and progress instead of leaving execution state implicit in conversation history.
- The model can invoke atomic tools or synthesize multi-step Python programs that group actions into inspectable execution units.
- Calibrated 3D measurements provide spatial information that is harder to infer reliably from raw images alone.
- Finger-referenced dual-arm control maps end-effector commands into a control space aligned with the robot's grippers.
- A single admission gateway checks every action, preventing the model from bypassing geometric, safety, and stage constraints.
The runtime follows a preview-admit-execute-validate-recover loop. Before motion, the agent inspects a rendered trajectory preview while simulation is paused. During execution, the harness checks declared stage tolerances and grasp bounds. If measured state diverges from expectations, it stops the robot and returns the discrepancy to the model, allowing in-place recovery rather than restarting the task.
Architecturally, the system treats robot actions as transactions with preconditions and observable outcomes: the model proposes a plan, the gateway validates it, the executor returns measurements, and the agent decides whether to commit, stop, or replan.
Engineering Trade-offs: Reliability at the Cost of Latency and Complexity
Auriga's gains come from multiple explicit checks, not from a free improvement in low-level control. Trajectory previews add rendering and model-observation latency. Python synthesis depends on model competence and a secure runtime. Per-stage tolerance and boundary checks increase the duration of each control cycle. For fast manipulation or irreversible collision risks, stopping and replanning may also be slower than continuing under a learned policy.
The system is tightly coupled to its robot configuration. Task-stage semantics, tolerances, grasp bounds, calibration, gripper kinematics, and control interfaces must be co-designed. “Task-agnostic” more precisely means that the harness does not require task-specific training or trial-and-error; it does not mean that hardware and geometric calibration require no adaptation.
Synthesized Python also expands the attack surface. Production deployments need process isolation, filesystem and network restrictions, deterministic module whitelists, resource limits, and audit logs so that generated code cannot compromise the robot control host.
Evaluation: Strong Zero-shot Robustness, but Comparison Conditions Matter
The harness was developed across 13 cross-task capability iterations and then frozen for evaluation. On RoboDojo, it reaches a 57.7% success rate and a 64.0 score, ranking first. The report states that the reference harness on the same model achieves less than half that success rate.
The scene-randomization result is particularly informative. Auriga remains first while learned policies degrade by as much as 33.9%. This is consistent with its architecture: trajectory previews, calibrated measurements, and boundary checks expose execution divergence directly rather than requiring a policy network to learn a new mapping outside its training distribution.
On RoboCasa365, the frozen harness reaches a 73.2% success rate and leads VLA-assisted agents by as much as 23.1 percentage points on composite tasks. These results still require careful interpretation. Task composition, prompt templates, robot assets, control frequency, retry policy, and confidence intervals can materially change leaderboard rankings. The abstract also does not clarify whether every reported episode excludes human intervention or environment resets beyond the stated absence of target-task trials. Independent reproduction should report ablations without previews, without recovery, and without program synthesis.
Deployment Guidance
Teams interested in the approach can integrate it in the following order:
- Introduce an explicit task state machine that separates goals, stages, and completion predicates from conversational context.
- Expose 3D measurements, gripper state, and collision checks as read-only tools rather than exposing raw controller APIs.
- Render a trajectory preview for every action and define deviation, grasp-error, and stage-tolerance thresholds.
- Prefer local recovery, with full replanning as a fallback, and record every deviation and recovery outcome.
- Sandbox generated code and enforce permissions, resource limits, and deterministic audit trails.
- Report planning success, action-admission rate, online recovery rate, end-to-end success, latency, and cost separately.
Auriga's main contribution is a transferable engineering pattern: a general-purpose vision-language model handles cross-task decisions, while a task-agnostic harness turns those decisions into measurable, stoppable, and recoverable physical actions. It does not replace learned robot policies, but it is a compelling architecture where control interfaces are complex, simulation-to-reality error matters, and zero-shot adaptation is required.
— END —Source: Hugging Face Daily Papers (2026-10-31)
Tags: #Auriga #Embodied AI #Vision-Language Models #Robot Manipulation #Agentic Workflow
Editorial & Fact-Checking Note: This article is compiled from primary research, official release documentation, and source papers by the ZICQ Newsroom pipeline with automated entity verification and human editorial review. If you notice any technical inaccuracy, please submit a correction via our corrections policy or email our editorial desk directly.
Community Comments